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la chapter III, an evaluation is given of the reliability of multiple 
choice tests, standard error of the passing score, reliability 
coefficients, and reliability of the writing test score. The fourth 
chapter describes score scaling and passing scores and offers 
recommenuations for the writing subtest. Chapter V discusses methods 
used to assure freedom from ethnic or cultural bias in the test. 
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PREFACE 



Four bulletins have been developed to -provide information about the Florida 
Teacher Certification, Examination. Bulletin I -describes the development of%he 
examination and presents tKe specifications for each of the subtests. It also 
pmvijies a list of supplemental materials and references and a set of sample 
Items to illustrate the kind of items which viU be used in the examination. 
Bulletin II presents a general description and specifications for each item 
in the reading, writing, and mathematics subtests. Bulletin III provides speci- 
fications for the professional education subtest of the Florida Teacher Certifi- 
cation Fxamination. It includes a description of the content base and Item' 
specifications for each of the professional education items to be Included in 
the examination. This bulletin, the Technical Manual, is the fourth in the 
series and describes the technical adequacy of the examination including such 
topics as establishing tes< reliability, test validity, passing scores, an^' 
protecting the test, from cultural or ethnic bias. 

It is expected that faculty members of teacher education programs 
students in these programs will be especially interested In Bulletin I. 
of Teacher Education Centers and school district staff development direc 
also find the information useful. - The specific item specif-icaflons and ot 
information in Bulletins II and III will probably be of. special Intere't 
professionals involved in program development and evaluation. Bullet 
designed primarily for measurement proft'ssionals. 

« * 

Please note that the scope of the examination is limited to the essential 
generic competencies which are assessable by a written examination. There has 
been no atten9)t to cover all aspects of teachier training. Hany important com- 
petencies are assessable only by direct observation, and many cwnpetencies are 
specific to the subject matter taught or the developmental level of the students. 
It is also in?)ortant to remember that teacher education Is dynamic; it must 
change to reflect- and incorporate new research evidence and wisdom acciimul ated 
from experience. For these reasons, even though the examination has been care- 
fully developed and reflects the current state of knowledge and priorities for 
the general preparatioi> of teacners. tte specifications for the examination 
should not be used as the sole basis for a teacher training program. 

The Department of Education encourager professionals to make recommendations 
to improve the Florida Teacher Certification Examination. Recomnwndati-ons and 
other Inquiries should be addressed tQ: 




Dr. Garfield Wilson, Administrator 
. Teacher Certification Section 
Florida Departirent of Education 
Tallahassee, Florida 32301 



Dr. Thcmas Fisher, Administrator 
Student Assessment Section 

Florida Department bf Education 
Tallahassee,. Florida 32301 
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CHAPTER I 
DEVELOPMENT Cff THE EXAMINATI(»I 

Section 23L17 F.S. retjulres that, beginning July 1, IW, each applicant 
for initial teacher certification shall dmnstrate on a ci^rehenslve witten - 
examination mastery of the mlnlRiiHii e<»i|}etenc1fts ailopted in Florida Statutes and 
State Board of Education Rules. In order to develop the Florida Teacher Certi- 
fication Examination (FTCE), the following U%U had to be acco«iH)lfshed: 
(1) planning, (2) writlftg and validating test Items, (3) field testing the. 
certification examination items, (4) setting passing scores, and (5) preparing 
for test assembly, administration, and scoring. The procedures by which each 
task was addressed are described In this chapter. * , 

The events associated with each developmental task are described In chro- 
nological order for a given task; however, since many of tte tasks overlapped, 
there is sme duplication of time lines across tasks. There Is also some 
redundancy in these descriptions, because a single activity was sometimes used 
to accoiT«)lish more than one task. The calendar ot key events Is provided In 
Table 1.1. Reference to this calendar will make clear the overall chronology of 
the development of the exanlnatlon. 

, Planning 

• The subtasks Involved in planning for the examination were (1) Identifica- 
tion and validation of the essential tea<;h<2r c(»^tenc1e$, (2) general planning, 
for examination development, (3) {fevel(^?ii»nt of sufoskllls for each conqsetency, 
and (4) development of test and Itan specifications. - Each of these activities 
is. described in this section. 

Identificatio n and Validation of the Essential Teacter' Competencies 

The single most important step In the develorant of an examination Is the 
determination of what it Is to cover. ¥rm 1975 through (^tober 1980, the 
Council on Teacher Education (COTE) was Involved In t»w itevelopment and iB^le- 
mentation of a "competency based model" of teacher certification, -which was 
enacted Into law as Committee Substitute for Senate Bill -549 (CSSB 549) and berame 
Section 231.17 F.S. During this time, COTE served as a statutory advisory council 
appointed by the State Board of Education to advise the Commissioner of Educa- 
tion on all matters dealing with teacher education and certification. ^ 

The "minimum essential c«i«)etenc1es," which were to be t»« basis of 
teacher certification in Florida, were Identified In a study conducted by COTE, 
beginning in Mav'ch 1975. The first phase of the study was the id«itif1cation 
of a set of competencies deenwd essential for all certified ^rsonnel. Follow- 
ing an intensive literature search, the Council prepared a survey Instrument 
listing 48 competencies and submitted It to a five percent ranitom sample of all 
certified educational personnel empl<#ed Florida for 1975-76. The survey 
rev(.'a1ed an extremely high degree of consensus on 23 of the 48 coaqjetencies. 
The 23 competencies identified in the survey were recowBffinded to the Conwlssion- 
er of Education for incorporation into State Board of Education Rules governing 
teacher education and certiflcotion. 



^Jh^ Council on Teacter Education was replaced in 1980 by the Florida 
Educational Standards Commission. 



TABLE 1.1 

CALENQAR OF KEY EVENTS IN Tl£ > 
DEVELOPMENT OF THE FLORIDA TEACHER CERTIFICATION £XANINATI(»it 

Identification and Validation of the Essential Competencies 1975-1976 

Development of the Coi^tency-Based Model of Teacher Certification . . 1977-1978 

SBE Rule, Requiring Es^ntlal CoBH>^teilc1es for Teacher 

Education Progr«8S ........ Narch 1978 * 

Section 231.17 F.S. Enacted June 1978 

Ad Hoc Professional Task Force Estabf isted . . . July 1978 

Development and Valldation of SubsklUs for Each Coi^tency . . July-Octbher 1978 

Task Force "Repprt to COTE \ Noveinber 1978 

COTE Recommendations on Test Deyelopraent by Contractors ...... January 1979 

Development of Item Speciflcationi . ' . ' March- June 1979 

Review of Itm Specifications by Task Force July 1979 

Refinement of Examination Specifications by 

Measurement' Consultants August 1979 

Develojhnent of Items by Contractors , , . September 1979- January 1980 

Expert Review of Items January 27-29, 1980 

Field Test of Items ........ April 17-18, 1980 

Analysis of Field Test Data April -June 1980 

Setting Passing Scores • • • ••'""^ 24-26, 1980 

Approval of Contract for Formatting, Administration and 

Scoring of Examination July 15, 1980 

Adoption of SBE Rules for the Florida Teacher 

Certification Examination July 30, 19B0 



» 



After extensive publicity and public discussion, the 23. Essential Generic^ 
Competencies vffiire incorporated in March 1973, into State Board of Education 
Rules governing the curricular requirements at institutions of l^^jfeer education 
with approved teacher ^educat ion programs. All of the- COTE delibamions, deci- 
sions, and recfflnmendations, along with the result.s nf the nun«rous surveys, are 
documented in .the COTE Minutes for 1975 through 1978. l^e competencies are 
listed in Appendix B. 

fagne ral Pl ann ing for E xami nation Development 

In June 1978^ Immediately after the Governor bad signed CSSB 549 into law, 
a meeting was called by the Department of Education to begin planning for the 
written examination requii'ed by the >aw. Attending this meeting were manbers of 
the COTE Executive CtBnmittee, representatives from the tao state teachers' 
organizations and the state aduinistrators* organization, and 'legislative staff 
numbers. This iwjeting resulted in rectsfflirendations to the Department of Educa- 
tion as to timeline, tasks, and professional involvement for the development of 
the tieacher certification examination 

. Responsibility for directing the development of the evamination was 
assigned to Dr. Garfield Wilson, Office of freservice Teacher Education, Depirt- 
ment of Education,' vrlth COTE serving as tlw primary advisory group. In July 
1978, a Professional Ad Hoc Task Force was convened. The Task Force consisted 
of six'classroom teachers, three teacher educators, one scfrool administrator, 
and one Ctepartment member. The classroom teachers- had been reconwffinded for 
participation by each of the two state teacher organizations. Dr. Jacob Beard, 
Professor of Education, Florida State University, was selected as a measurement 
consultant to assist COTE and the Task Force. ^ 

The Professional Ad Hoc Task Force was responsible for many activities in 
the early development of the examination. One of its first duties was to con- 
sider the overall design for the examination, an activity conducted with the 
assistance of Dr. Beard. The Task Force difcussed major design tasks and prob- 
lems and then ccwnpilect. ^ of rfecommendations which were circulated for review 
throughout the State. Cownts were solicited from all teacher education insti- . 
tutlons, school districts. Teacher Education Centers, Ctepartnwnt staff, and, pro- 
fessional ,ornanizdtions. 'All reactions and comn^nts wefe carefully considered 
by Department staff an-l the revised reconinendations were then presented to the 
COTE Committee. 

At its January 1979, meeting, COTE considered the reconwitendations devel- 
oped by the Department, it subsequently adopted thp following reconwendations 
concerning the overall dfesign parameters for the teacher examination: 

i^The Ltepartment also was assisted in the planning phases of this, project 
by Dr. Howard Stoker, Florida State University, Dr. Annie Ward, University of 
. South Florida, and Hr. Robert Feinberg, University of Florida. 
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1. The test should be an objective-based paper and pencil 
exsnl nation. 

' 2. The test should be organized Into tw> sajor parts addressing* 
general educ&itlon and professional education coiiq>etenc1es 
with each part divides: Into subtests as follows: 

* 

General Education 

A. Writing 

B. R&sdlng and Listening 
C* Mathematics 

Professional Ecfaicatlon 

A. Physical, Social, Academic Oevelopinent and Inter- 
, personal Skills 

B. Tec^lcal Skills 

C. ' AAslnlstratlve Skills 

3. The response foraat should Include: 

A. Writing— a writing prochictlon task that will be 
rated hollstlcally by selected evaluation experts 

B. fading Cwrehenslcm— (1) A "Cloze" test . 

Iz) Reading passages with 
mjltlple-cholce questions 

C. ' r^ltlple-choltt or sm otl^r forw of object1y,ely 

scoredtl terns for other subtests 

4. Specifications for test Items should be based on the subskllls 
that are reconwended by COTE after consultation with experts. 
The subskllls must be aisenable to paper and pencil testing. 
The 127 subskllls for tte 23 essential generic comj^tencles 
and the competencies specified In statutes were identified by 

Ad Hoc Task Force In July 1978. 

5. Specifications for the test Items for each subtest should be C 
developed on a contract basis with Florida Institutions of 
Higher Educ^^tlon with doronstrable expertise In. the arj?a In 
quest Idn. 

6. Specifications developed by contractors should be carefully 
reviewed by COTE and the Task Force. 

7. Following approval of specifications by COTE, the State should 
develop a precalibrated item pool [calibrated through Rasch 
technology], 3 

8. The item pool should be large enough to allow construction of 
a new form of the test with each a<fai1n1 strati on for purposes 
of security. 



3Minutes of the Council on Teacher Education, January 1979. 
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9. The items jn the precallbrated item pool should be included on 
appropriate subtests in rough proportion to the median weights 
Assigned by the Task Force. : 

10. . The test wtll be a requlreitent for eligibility for a Florida 

teaching certificate; it will not be a requirement for a 
baccalaureate or other degree. • 

11. The test fee should be sufficient to cover the cost of mainte- 
nance and adninistratlon. 

m 

12. The test should be acfaninistered at least twice a year at 
selected locations statewide. 

13. Only pass/fail scores shall be reported on each subtest. All 
subtests must be passed; only subtests failed must be retaken 
.or eligibility for ne^lar certification. 

14. The procedure for establishing cut-off scores should be devel- 
oped by measurement experts, advertised, and incorporated into 
$tate Board of Education Rules no later than April 1979. 

After much discussion by COTE, tte Department, and various consultants, a 
decision was made to issue a Request for Proposals (RFP) to the various univer- 
sities in the state for the development of item specifications and items for the 
examinatlob. tt was also decided to issue an RFP to universities and conmercial 
testing agencies for the operational tasks required for the formatting, printing, 
field testing, administering, and scoring of the examination. 

Devel opment of Subskills for Each Cftnpetency 
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At the first Task Force meeting in July 1978, one assignment was to devel- - 
op subskills for 22 generic conpetencies (competencies #6 and #19 of the original 
23 essential competencies were combined by legislative action). Dur'.ig an intensive 
week of writing, review, and revision, the Task Force developed 127 subskills as 
indicators for the essential competencies. 

In August, 1978. the subskills »«re submitted to a randwn sample of certi- 
fied personnel in the State to determioe how well the subskills were related 
to the original competencies. All but 12 of the subskills met the criterion for 
acceptance. Two of the subskills {5d and 14b) which did not meet the criterion 
were retained because of statutory mandates. This left a total of 117 validated 
subskills. Of these, twenty-six (26) were distributed across the five general 
education competencies, and the remaining 91 were distributed across tl« seven- 
teen professional education competencies. (See Appendix B for the list of 
competencies and validated subskills.) 

The competencies and subskills were reviewed by COTE at its September 1978 
meetinq. COTE was asked to make some recommendations on the most appropriate 
evaluation procedures for the competencies. COTE recomnended that since the 
Task Force had developed the subskills it would be the most knowledgeable group 
for developing recoamendations on methods of evaluation. 
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In October 1978, the Tas^ Force' was reconvened with measurement consul- 
tants Dr. Jacob Be«rd and Dr. Annie Ward assisting the group. The Task Force 
in a two-day session, developed recoiwiendatlons fbr the most appropriate eval- 
uation procedure for the essential competencies and subskills. These reconmen- 
dations ind.icated t'r« .competencies and subskills that could be measured most 
appropriately by each of three procedures: a comprehensive written (^neric) 
examination, a written specialization examination, or a performance evaluation. 
(See Appendix B. } 

t 

The Department of EduMtion staff, with assistance fr«n Dr. Beard, pre- 
pared draft overall test design specifications for the examination and the 
t subtests. The proposed specifications were reviewed by COTE at its September 
and November meetings. The proposed specifications, including the list of 
validated subskills, were distributed to all cfiljeges and universities with 
approved teacher 'education programs an** *o statewide professional teacher 
organizations for review and critique. The reactions from these groups were 
reviewed by the Department and revisions were macte as needed. 

Development of Item and Test Specifications 

In kt'ping with the January 1979, reccwmiendations of COTE, an RFP was 
issued, and in fiarr^i 1979. contracts Were awarded for the development of item 
specifications v^'-:'! would guide the creation of specific test questions. The 
contracts awarded were as follows: f ^v. 

Writing (Competency 2) 

Florida State University 

Dr. Gordon Brossell and Dr. Janms Hoetker 

Project Directors 

Reading and Listening (Competencies 3 and 4) 

Florida State University 
Dr. F. J. King 
Project Director 

Mattematlcs (Competency 5) 

Florida State University 
Dr. Tom Denmark 
Project Director 

Professional Education: Personal, Social, Academic 

Development (Competencies 6/19, 

20, 21. 22. 23) * ^ 

, Florida State University 

Dr. Gary Peterson and Dr. Steven Roll in 
Project Directors 
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Technical Skills (Cowtencles 7, 8, 9, 10, U, 12, 

13. 14) 

Florida Atlantic University 

Or. Wlllard mm* and Or. Anna Helson 

Project 1D1 rectors 

A«ta1n1strat1ve Skills (Cca^etencles 15.-16, 17, 18) 

Florida Intematlwal University 
Jfr. Colleen Ryan 
^ Project Director 

All contractors i«re required to have their draft itm specifications re- 
viewed by experts f row other Instltutliws. A smp\e of the specif Icatlww was 
also reviewed by COTE at tl» Itey 1979, aeetlr^. In July 1979, the Task Force 
was- reconvened to iwlew and critic^ the 1t«B specif Icatlwjs. The Task Force 
also recoiwemted priorities for the Inclusion of tl« subskllls for proffesslonal 

education co^tencles (#6 thrwigh #23) on tj^^^'^t «nJ-tJ>'22fIl!*S'^tSlr 
the written examination. Seventy-two (72) of the subsklin were rated.e1ther 
"essential" or "^rtant," idille the rest i«re rated "not necessary. ^ 

The Task Force spent three days reviewing and critiquing the item 
specifications and prioritizing the sobskllls. Coinaents and sug^stlons for 
improving tN Item spec4f1cat1ons were coeq>11ed. 

During the prwsss of reviewing the it«ji specifications, the Task Force 
also considered the general test specifications and furtf«r refined the^ It 
was recommended that Cw^tencles 1 (oral <?»«n^cati«i) and 3 JUf^f i»9> ^ 
should be deleted from tl» written examination but sNxild be assessed by per- 
formance evaluation during the year-long internship. 

In August 1979. a full day work session was held with DK staff «nd 
measurement consultants Dr. Jacob Beard, Dr. Howard Stoker, and J^-^i^ "fjj- 
The purpose of this session was to review and refiiw the-pirwsed specifications 
for the exOTlnatlon. After reviewing the 1t«» specif Icati^s, Task Jjr^ rwwn- 
mendatlons/and previous COTE recommendations, tl« measurewwit consul jants made 
the following tentative recwsiendations about t;he B»ke up of the examination: 

1. The examination should consist of ftxir subtests: Wr1t1?»g, 
Reading, Hathe^tics, and hrofessional Eihication.a 

2. The Writing exfflnlnation should be 45-60 minutes in length, 
including time for reading and ctoosing between topics. ^ 

3. The Jteading examination should be approximately 30 minutes 
In length and Incluite a maxiswrn of 100 Hem in a modifi^l 
cloze format. 



4Most of those rat*»d "not necessary" were judged to be covered In other 
subskllls. 

BThree professional education subtests had been reconnwncted by COTl 
January 1979. 
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i The Mathenatics exoBlnatlon should consist of approximately 
40 items with one hour of testing time. 

■ 5. - The Professional Educatlwj examination sl»u1d consist of 
. applGOxImately 100 iteias with % hours of testing time. 

The consultants also recOT«ended that tl» exwilnatlon should 1* adnlnis- 
tered on Saturdays four times cbiring the first year SS^J;*^.';*^ J^^^J"* 

locations througtout the state; that the exwoination s^ld he sc>«<^Jf<*^tor a 
full day, with lithe«atics. reading. «id siting in the ^"'^^ ,5"^,*^ P^J^f" 
slonal edi«:at1on exa»inat1on, in tf» afternoon; that scores stould be r^^rted 
on each of the subtests; andHhat persons scheduled to yeiaje^;^ ^^^^ 
the subtests should be tested during a regularly sclssduled testing administra- 
tion. ■ " 

In September 197^, the Department entered into a contract with" the. lAil- 
verslty of South norlda to provide psyctRSsetric assistant to tte examination 
J%jeci l^. Lie «ard was assigned this res^^^ ^^^n'jSTMJii 
Assisted the Departpient in the P">*Eti«i of the exajination 
tions describing the exsnination. Dr. terd ^IJfl^i^^.fPf^^!*^^ * 
the examlnatiofl and for each subtest and <^^^el J.^^^^®!^"? J^'JjJ JJ?" . 
sented the test and ite» specifications. TNse bulletins are: The Flprlda 

Writing and Validating Test Itm^ 

In order to generate a pool of validated test Items It was necessary to: 
(1) create the test Items. (eVpnot test the items, and (3) « S^J^^" 

of the 1t«BS. The procedures used to accomplish each task are described In 
this section. 

Creating the Test Items 

The Department negotiated with University personnel In Florida for revising 
some of the Itan specifications and for writing items n several areas. The 
^ractort ^ prSared the item specifications were Invited to fubm t a proposal 
S? wItiS 1ton?lccording to thosrspeclficatlons. The Technical Skills area 
tealTs! 9. lO.'u, 12. 13. and 14) was divided Irtto two Par^; Jhe 
specifications contractor for that area agreed to prepare Items Jo^'.^o^f ^^J^^®^. 
7, 12, and 14 and a new contractor was secured for coB^tencles 8, 9, 10, il. ana u. 

Contracts were awarded as follows: * 

Reading 

Florida State University 

Dr. F. J. King, Project Director 

21 test passages, each containing 10 Items 
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» * m 

m 

Dr. Gordon Srossell, Project Dlt^tor 
Val1dat1<m of ts^iv and eva1uat1<m of three 
presentation 'ii»«tes 

Nathenatlcs X 

Florida State iftilyerslty 

Dr. T<Mli Deffisarfc, Project Director 

102 natltematlcs items 

• 1^ ^ 

Florida State University 

Dr. Gary Peterson, Project Director 

1(K) IteRS for coRipetencles 6, 20, 21, 22, and 23 

(Personal,' social, and acaitenli: development) 

Florida Atlantic Unlvi^rslty 
Dr. Xlllard Nelsm and Dr. Anna Nelson 
Project Directors 
f 80 Items for cosqsetencles 7, 12, and 14 

(Diagnosis and assessment) 

Florida State University 

Dr. Itolter Wager, Project Director 

80 Items for cffiq»etenc1es 8, 9, 10, 11, and 13 

(Teaching and learning) 

University of l^st Florida 

Dr. Ronald PeaMa, Project Director 

99 Items for c^>etenc1es IS, 16, 17, and 18 

(A(^1n1strat1^/classrocss mana^ment skills) 

/ * 

In addition to writing Items, contractors were required to arrange a series 
of reviews to establish the relatedness of the Itei^ to the validated competencies 
and to eliminate obvious technical pyt)bl^s. These review processes Inducted: 

1. Internal review. • All 1t«ns were reviewed by the mitractor's 
staff before the; Items were approved. 

2. External review/ Each Item was reviewed by a minimis of four 
experts. They included cl assroaiHtfeachers , content specialists 

either un1vers;tty or public school), and measurement speclal- 
sts. Each grmip also IncliKled both sexes and r^resentatlves 
of the m^QT ethnic groups. 

3. One-on-one adslnlstratlon. Each Item was aitolnlstered indivi- 
dually to at least three perscms, with the ex»i1nee solving the 
Item aloud. The examiner noted the types of problems exam- 
inees had with each item. 
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After the contractors conpTeteB their review and revision of the Items, 
the Items were submitted to the Department for review and critique. The Items 
were further revised as needed and prepared for the pilot testing. • 

Pilot Testing the Items 

The contractors conducted a pilot test for tteir own Items, sorting the 
Items Into, two or three forms for convenience In achilnlstratlon. Common Items 
were t>rov1ded for the various forms of the pilot tests so that they could be 
calibrated on a common scale. The ssm^le of students who tooj? the pilot tests 
was made up of seniors at teacter education Institutions df the state. The 
Institutions In which the pilot tests wej?e conducted are Indicated below. Each 
fonn of each subtest ms taken by at teiftt 20O students, except for the leading 
it, where the san^Jles Wire 50 for each foo. ' ' " ' ' 



Reading 



Writing 



MatheMtlcs 



Florida State University 
Florida A & M University * 

. Florida State Ifcnlverslty * 
University of South Florida 

Florida State Ifculverslty , , 
University of Central Florida 
Florida A & M University 
Bethune*Cookman College 
University of Miami 
Jacksonville IMIverslty 

Professional Eductfflon (Personal, Social, anJi Acadenic Development) 

Stetson University 
University of Jtorth Florida 
University of South Florida 
University of Vtest Florida 
BetNine-Cookman College 

(Diagnosis and Assessment) ' 

Florida Atlantic University 
Florida International University 
Jtova University 

(Teaching and Learning) 

Florida State University 
University of Florida 

(Actalnl strati ve/Classntom Management Skills) 

Stetson University 
University of Florida 
University of South Florida 
University of West Florida 
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DaU from the pilot tests were analyzed to secure traditional Iteia 
statistics. In addition, a contract ms negotiated with Florida State Uni- 
versity to analyze the data using the Rasch model. Data from both these ^ . 
analyses were available for the final review by experts. 

• Reviewing the Test Itews ^ 

A panel of experts was assend>led in Tallahassee on January 27. 19B0. to 
review the Items. There were three asc»cts of the review process: content 
validation, technical <^11t1es, and absence of bias. The review teaa consisted 
of classroom teachers, school aftalnistrators and supervisors, teacher e^icators, 
and m^surement experts. The reviewers had been selected in a way that provided 
expertise in all areas of the ex^iinatlon, as well as expertise In msurenent, 
linguistics, and bias. Three consultants hel|»d to plan and comfeict tN review 
process: Dr. Milllaa Hehrens. Michigan State Iftiiversity; Or. Howrd Stoker, Rorfdft 
State University and ETS; Dr. Toh Frejio. forraerly at diversity of South Rorlda, 
now Pi^esldent of PDE Associates. The coaplete roster of participants Is Included 
in Appendix A. The review procedures are ttescribed in Chapter 11, and the agenda 
is presented in Appendix D. 

'Field Testing of the Certification Examination Itews 

The field test of exawination items was conAicted by the Student Assessment 
Section. Division of Public Schools, Department of EAicatlon. All items which 
had been accepted by the review panel, or which tad b^ edited, revised, or 
rewritten according to recomnCTdations of that panel , were assembled into seven 
field test forms. Severa-l ccxaaon items were provided for each subtest to be used 
as linking (anchor) items. The vanious forms of the test booklets were randomly 
distributed to the examinees. The field test mbs conducted on ^ril 17 and 18, 
1^0, with 1,186 examinees at i4 sites located throughout Florida. The sites were 
selected to represent public and privite schools, schools of varying size, and 
schools with varying ethnic populations. 

Most of the examinees were students were cOB^)let1ng a teacher educa- 
tion program am! were completing their internships. In acfalltion, a small group 
of non-degree Vocational/Industrial teachers who were con^Jleting their profes- 
sional preparation were included. Participating institutions ware asked to 
try to secure total participation of all eligible students. However, at some 
institutions, the tem had encted and many students were m> longer available; other 
institutions had interns scattered over the entire state and coBiminlcation with 
some of them was difficult. 

luring the field test, a few participants were unable to complete 
all the items. Tterefore, the analyses were limited to those exAuined who 
completed each subtest. Since all test booklets were arranged so that Read- 
ing items were first, then Mathematics, and finally. Professional Education, 
most of t)w Incomplete 'papers were In the Professional Education area. The number 
of cases Included in the analyses Is shown in Table 1.2, 
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TABLE 1.2 

t 

PARTICIP/WTS IN FIELD TEST 





Fom mmber 




1 


2 


3 


4 


5 


6 


n 7 


Total 


Reading 


170 


172 


168- 


171 


m 


167 


170 


li86 


.Mathematics 


170 


172 


168 


pi 


168 


167 


170 


1186 


Prof. Educ, 


164 


166 


158* 


164 




161 


166 


1141 



The Department of Education data center scanned the answer s)»ets and 
provided distributions of scoues^or the total group as well as for the various 
institutions, progrw areasr and sex and etNilc groups. In addition. Dr. Jacob 
Beard at Florida State ttiiversity provldetf'Rasch analyses. Item-total and inter- 
Item correlations, reliability indexes and bias analyses. The analyses are re- 
ported in later sections of this namial. 

Setting Passing Scores / 

Passing scores for each subtest i«re determined by a panel of twenty-one 
judges, a'l of ^tmt were either current or past meabers of COTE and *^ had been 
involved in the development of the ex»ii1nat1on. 11« panel was naite up of eight 
classroom teachers, five school adninistrators, five teacher educators, and 
three conimmlty represantatives. (See Appendix A for a complete roster.) 

/I 

The procedure used for the Reading, Hatheniatics, and Professional Educa- 
tion subtests involved Identifying Items which represent^ the level of perfor- 
mance which was acceptable. For tte Writing subtest, various levels of pre- . 
scored wrtting samples were consittered. 

As a result of these deliberations, passing score recoim«ndations were 
made to the Comnlssioner of Education for each subtest. These reconroendations 
t#ere adopted as a rule by the State Board of Education on July 30, 1980. The 
procedures are described more fully in Chapter IV of this bulletin and In the 
minutes of COTE. June, 1980. 

Preparing for Test Assembly, Administration, and Scoring 

The operational tasks of preparing test forms, administering the tests, and 
scoring the answer sJ»ets during the mo-81 school year *«re conducted by the 
Office of Instructional Resources (OIR), University of Florida. A «mtract fdr 
these services v»s awarded through a co»i9>et1t1ve bidding process. OIR was respon 
sible for assisting the Department conduct the first three test actainistrations. 
These actainistrations, which took place at several different sites across Floriite 
are described in other publications and reports prepared by the Department of 
Education. ' 
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CHAPTER n C 

vAtiom 

fs concent idth Infermices ai^ropHately frm test sc;ores* Hiese Infer* 
ences be related to <l} i«hat is teln9 msured by a test an^ (2) what can 
be Inferred about other benavlor, 

Jaeger (1979) classified the tMo'tjfpes of Inferences as tliose which relate 
to a well -defined Main of tasU (proximal) and those i^lch relate to sonie 
ultimate criteria outside the sai^ied doiMtn (distal). The^fonaer Is of Imwr* 
tance to the Florida Tmher Certification Exanlnatlon because of the wording of 
Section 231.17 F.S. 

• Section 231.17 F.S. provides that "teglnnlng Juljf 1» 1980 .... each appli- 
cant for Initial certification shall deonnstrate, on a a>iiq>rehens1ye written ex- 
amination and through such other procedures as nay be specified l^r the state 
board, laastery of' those orinlfiaip} ess^tlal gmHc and ^leclalliatlw appetencies 
and other criteria as shall be sdoptSA Into rules by the st&te tioard Note 
that the statute addresses only the status at tine of certification, which Is a 
proxiwal concern, and does i»t require that any inferences be about the 
distal concern. I.e., tetter the test Is directly related to the excellence of 
teaching. The dofoaln of Interest was that of eitry level skills. 

Furthemore, validity, accofdlng to Crmibach (1971), relates to whether 
or not the test Items adei^ately sa^fe the "tml verse of tasks the developer 
Intended to measure For the Florida Teacher CertlflcatliHi Exaalnatlbn, 

the broad dimensions of the Intencted fhmin were Identified In Statute 231.17 
as follows: • 

1. The ability to write In a logical and understandable style with 
appropriate gnSnmar and sentence structure; 

2. The ability til. coi^rehemi and Interpret a message after listen- 
ing; 

3. The ability to read, comprehend, and Interpret orally and In 
writing, professional and other written isaterlals; 

4. The ability to comprehend and work with fundamental mathematical 
conceptsrand - 

5. The ability to cffi^rehend patterns of p)iys1cal, social, and 
acattemic development In students, and to counsel students con- 
cerning their r^ads In these areas. 

The extent to which the Florida Teacher Certification Examination provides data 
^ich allow Inferences \o be maite about the% dimensions may be (tetenslned by 
examining the adequacy of the set of operations used In test construction. The 
steps through which the Flor1<fo Teacher Certification Examination was developed 
are described In this chapter. 
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Oeveloiment of the Itms ^ 

Tt% Intended coverage of the examinetlon tas (tetermlned by a process t«h1ch 
used professional consensus *to (1) Identify co^Metencles idilch srmuld be deiRon- 
strated as a condition of certification* and (2) Identify subskllTs associated 
with each cm^t&nc^. The procedures by which the intenited covera^ .was litentl- 
fled Included surveys of the profession, reviews by COTE, reviews by the ad 
Task Force of COTE, and reviews teachers and otl«r educational $>ersonnel. By 
th^se processes, a set of cffi!9>etenc1es and subskllls wpre deve1iK)ed which defined 
the doiiain to be covered by the examination. The ttoaaln was furt)»r explicated 
by the various contractors when they prepared the item specifications. 

The specifications for^each subtest and for Iteas of the Florida Teacher 
Certification Examination have be«» pubTIsM in three Inilletins: Bulletin I; 
Overview ; Bulletin II; Tl» general EcfaicatitH^ Subtests - teadlng. Writing. >atbe- 
iiHcir "BuT!et1n tth Tt« Professional rducatiwr Subtest. These bulletins were 
aistrTbuted to all Florida teaclsr e«ication Institutions and school systew 
personnel offices In the fall of 1979. An overview of tf» coverage of the exam- 
ination is provided In Appendix C. 

For the Reading subtest, the ikmin of material s was derived from materials- 
cownonly in use In FloridTteacher eAicatlon Institutions. The materials and the 
procedures used to identify th«fl are described In Bulletin IT. 

For Mathematics, a survey >as made of teachers and other educators, to 
(1) identify subsubskllls for the cos^>et«ic1es and subskllls and (2) determine 
the relative weights of oth^r dln^nsions of the test. The specifications derived 
from these surveys «re published in Bulletin II. 

For the Writing subtest, it was necessaf7 to determine (1) the kinds of topics 
to be used and (2) the ^thod of presentation of the topics. A stu^y was conduct- 
ed to resolve these Issues and tte results are presented In a report to DOE by 
Brossell (1980), "Validation of Topics and Co8|>arison of Three Presentation Modes 
for the Writing Subtest of tte Florida Teacher Certification Examination.*' The spec- 
ifications for the writing subtest and scoring standards are included in Bulletin II. 

For the Professional Education Subtest, contractors identified specific 
rules, principles, theories, etc., in the professional literature **J1ch serve as 
the basis for both the specifications and the it«is. \Thi8 body of supplemental 
raaterlai-was canblned, analyzed, and organized Into the "content base* to which 
all specifications of t»» Professional EAicatlon subtests have been referenced. 
Both the specifications and content base were published in Bulletin III. 

After the intended covera^ was Identified and explicated, items were writ- 
tcA and pilot tested and then coiqsared once again to the skills being measured. 
This last item review step Included (1) a check that each item addressed the in- 
tended competency subsklll and specification and (2) a check that each Item was 
free of the Invalidating effect of bias or technical problems. 
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Final test. Item f^leM 

The final Mm rev few Mas carried out In tw steps. First, Hess were 
reviewed by a special review panel which assaabled In Tallahassee on January 27-30 
1980; then the Itess were field tested tdth a repra^tatlve smsfitB of seniors In 
teacher education prograpns and revleved once again Oepartfflent of EdixatliHi 
staff. 

The review panel was coaiprlsed of classrooB teachers, teacher educators, 
and adiilnlstrators. Reviewers were selected to pm1<te covertge of all content 
areas of tim exanilnartlon and to Include women and nliwrltl^s. (See Ppi^Ain A 
for roster.) •> ' 

The review process was br^Ml^ed In^^MO tasfcs^ - Tasit I rlncluM the 
following activities: 

1. Keying the Item, to dieck tte accuracy of keyed respmse^ 

2. - ^Bllnd" Traceback as follows; • 

^ (a) For Kathematlcst tracing the Item to the suhsklll 
and tlw other speclflcatlm cate^rtes 
(b) For Professional S^icatlon. tracing the Ite to 
the ooi^tKicy, subsklll, and cont^t cate^ny(les) 

3. Rating the Item for appropriateness 

The Items i4re stratified 1^ test area, ftevleuers were also stratified by 
area of expertise. Items were assigned to reviewers in such a way that the 
following requirements were net: 

1. All Items were keyed and "traced back" by at least three 
specialists In the area of cont^t 

2. Each Item was rated for appropriateness fay at least five 
educators spreadacross various specialized grcMips 

3. No reviewer was l^ulred to trace back more than 60 Items 

4. Each reviewer also keyed and rated some aiMltlonal Items 
as to appropriateness 

The second task Included a review of Itoi content, bias, and technical (^allty. 

Committee assignments wre maite according to tte special qualifications of 
each reviewer pn6 to the needs for each type of expertise. Review forms were 
developed to guide the review procechire. All review l^rms are Incluited In 
Appendix E.2. 

Content Review 

The content reviews were competed by four panels selected to represent 
specialized areas of the test. Items were also sorted Into four groups. Ihe 
assignment of Items and persons to groups Is Indicated In Table 2.1. 
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TABLE 2.1 

HEN m 9mm i^siemm 

m EXPERT REVIEW 



Test Area 



Itevleif Specialists 



MatKe<nat1cs 
(102 I tens) 



Nathematlcs Teachers and Professors 
Hathenatlcs Educatlcm Professors 
Mathmtlcs Supervisors 



Professional Education 
A. Assessiaenty diagnosis, 
measurmnt 

Coinpetencles/Subskl lis: 
6b; 7 (all); lOh; 14 (all); 
18 (all) 

(approximately US Items) 



NB8»irei»nt Professors 

Supervisors of Testing and Evaluation 

Neasurement and Evaluation Consultants 
Classroom Teachers 



Professional Education 
B. Teaching and Learning. 

Netfiods, Naterlals, 

ProceAires 

Coi^>etenc1es/Subsk1 1 1 s : 
9 (all); 10b, c, d, f; 
11c, d, e; 12 (all); 
13 (all); 15 (all); 
16 Call); 17a, b. e, f; 20d 
(approximately 130 Items) 



Currlcultmi Mettwds Professors 
Curriculum Supervisors 
School Administrators 
Classroom Teachers 



Professional Education 
C. Personal, Social, 
Academic Development 

Competenci es/Subskll 1 s : 
6a, d; 10a, e; Ub; 17c, d; 
20b, c; 21b, c, d; 22d; 23b 
(approximately 124 Items) 



Education Psychology/Guidance Professors 
Guidance Counselors, School Psychologists^ 
Pupil personnel Supervisors 
Classrown Teachers ,^ 
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Each panel member was given a Sf»c1«ny desired review foro on »rti1ch to 
record his or her cossnents. Separate review foms mre provided for the Mathe- 
inatlcs and Professional Education subtests. The forms required that the items 
be rated as to adequac;^, relevancy, behavior re<^1redt and general Item 
characteristics. 

Bia s ftevlew 

The panel to review the Items for potential hias was cos^sed of slnorlty 
persons, wowen, and experts trained In linguistics. Many of these p$ople were 
experienced Item revlewsrs; ail were lcnowle<^ab1e about education. Review 
forms were prepared which required the reviewers to determine that each Item 
(or passage, In the case of the Reading subtest) was free of the appearance of 
bias and of biasing elements. (See Chapter V for additional Information on the 
bias review.) 



Technical R eview 

Panelists for the technical review Inc'uded both measurement and language 
arts experts. Both groups exsolned the actULl 1t^; In addition, tto measure- 
ment experts considered Item data from the pilot tests. Coimmily accepted 
guidelines for mult1ple*cho1ce lt^ and review fd«0s ^bodying the ^Ittellnes 
were provided to each panelist. Experts also noted any tecnnlcal defect not 
covered on thi form provided. 

Each aspect of the Task 11 reviews las conducted Indepenitently. As the 
reviews *«re completed for each subtest area, the project staff collated the 
Task I reviews and the three Task II reviews and made a decision as to the next 
step for each Itan. Possibilities Included (a) accept the Item as Is, (b) refer 
the item tO the re-write team, ^nd (c) return the Item to the contractor for 
replacement. 

^ The third task was the rewriting and re-review of 1t«ns. Some reviewers 
were assigned to rewrite teams to work with the contractors to revise the Items 
as needed. Contractors lere responsible for producing Items In a final form 
which were acceptable to the reviewers and to tte Department of Education. 

Analysis of Field Test Data 

The final step In validating the Mm", ms the review of field test data. 
(See Chapter I for a description of the field test procedures.) Itans which did 
not perform well (i.e., which did not "fit" the Rasch Model; did not wrrelatc 
well with other Items, or which did not discriminate well) were removed from the 
item bank. These Items were either thrown out permanently or revised and 
suh-jected to another field test. Staff of the ttepartment perfonned tMse tasks 
during the sunroer and fall of 1980. 

Stminary 

The validity of the Florida Teacher Certification Examination has been well 
established. The validity rests primarily upon the content validity of the test. 
No claim is made that the test scores will have predictive validity— I.e. , 
be able to predict the success of a prospective teacher In an actual class- 
room situation following passing the examination. It only Is claimed that the 
tests adequately measure the skills for which they were designed. This validity 
rests upon the manner In v^lch the tests were designed and the care and detail 
with which the, content area drawlns and test 1t«Bs have been itescribed. 
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CHAPTER in 
' REllABItTY 



ftellablUy may be ^flned as the ^ree of consistency between t«fo measyres 
of the sfiBRe th1ng.VT>» reliability a»is1&r9t1ims for the norlda Teacher Certi- 
fication Examlnatfon^imiftlple choice subtests (iteading, Ilathe»at1r«, sitd Profes- 
sional Education) are different €h«i those ef the tirltlils test, a production writing 
sample. With the imiltlple cN>1ce tests the pM^ry omcem Is for the reliability 
of an Individual's score, while for the Writing test, the concern Is for the 
reliability of the judges' ratings. The data for the reliability studies for the 
multiple-choice subtests Mere obtained frois tfm field test con^cted In April, 1980. 
(See Chapter I for a description of field test procedures and li*s for each form.) 
Reliability data for the Writing subtests were obtained In a stu4y conducted for 
the Florida OepartiseAt of Ethicatlon by Brossell (im), ''Val1dat1<m of Topics and 
Cos^arlson of Three Presentatlm Modes for Urn Writing Subtest of the Florida 
Teacher Certification Examination." 

Reliability of Multiple Choice Tests ^ 

An Individual's score on a test Is mde up of the *'true score" and an "error 
score." The true score niay be thmight of as the "domain" score. Since It is not 
feasible to actually give an Individual all Itens of a g1^ dt»a1n. the doaaln 
Is sampled to prepare alternate forms of a test. If an Indlvlchial took several 
forms of a test, all constructed by soling from the defined domain of Items, 
scores on the various forms of tt» test wmtld not vary except as a result of 
randCRi errors. Such random errors come from Item sampling errors and from changes 
in the Individual from one test to another— attention. Interest, fatigue, etc. 

Standards for Educational and Psychological Tests (APA, 1974) states that 

" 'reliability coefficient* is a s^erlc tenn. IFcan be based on various 

types of ev1<tence; each type of evidence suggests a different meaning." Relia- 
bility evidence is generally of two types: 

1. Intarnal consistency, which Is very Imjwrtant If items are 
viewed as a sample frtmi a relatively homogeneous universe. 

2. consistency over time, which Is very la^rtant for tests 
which are to be used for repeated measurement. However, 
the possibility that scores will change as a result of 
developmental or educational Influences must be considered. . . 

For the FTCE the important reliability Issue is the extent to urtilch an 
individual's perforn«nce level would be Judged the same regardless of which form 
of the test is taken. This is an internal consistency question since applicants 
win not ordinarily take the examinaticwi more than one time unless tl«y nave 
failed it. In that case, they may have taken additional preparation directed at 
improving their per*formance, so a difference would be expected between scores 
made from one time to another. However, It is essential that an Individual's 
score at one point In time be consistent with his or l»r score on another form of 
the test taken at tte sane tlme^oi; ^'Ith no intervening learning. That is, an 



individual's perfomance can be generalized to his or her performance on the 
domain covered by the test. Ttils type of reliability is assessed by nwasures 
of Internal consistency. Tw types of reliability estimations are reported 
for the field test data. First, the stfandard errors of scores at the passing 
score level of each subtest are examined, and second, the coefficient alpha is 
reported for each forih of each subtest. 

Standard Error of the Passing Score 

The standard error of a test score nay be used to identify limits that 
have a defined probability of including the true score. For example, the prob- 
ability is approximately 68« that an individual's true score lies within the 
range of the observed score ±1 standard error, and the probability is approxi- 
mately 95% that the true score lies within the range of ±': standard error from 
• the observed score. A reliable test will have.small standard error values. 

For the FTCE the decision point Is at tte passing score level, as estab- 
lished by State Board of Education Rules. Therefore, it is essential that the 
examination be very reliable at the passing score point on each subtest.- 
Table 3.1 reports the standard error for each field test form of each subtest 
at the passing score point. 6 

TABLE 3.1 

STANDARD ERROR FOR FIELD TEST 
¥0m OF THE FTCE EXPRESSED AS LOGITS 



% 


Field 
Test 
Length 


Passing 
Score 


Form Number 


Median 


1 


2 


3 


4 


5 


« 1 


7 

1 — » — 


Reading 


50 


l.,40 


.36 


.36 


.37 


.36 


.35 


.36 


,36 


.36 


Math. 


27-28 


1.00 


..46 


.47 


.45 


.46 


.45 


.47 


.46 


.46 


Prof. Ed. 


69-70 


\ .25 


.27 


.27 


.26 


.27 


.27 


.26 


.27 


j .27 



These standard errors are quite acceptable. Furthermore, since the field 
test forms for every subtest were much shorter than they will be on the actual 
examination and since some items *«re Included on the field test which did not 
meet acceptable standards for inclusion In the actual examination, these standa 
errors are expected to be smaller for the actual administration. 



^Passing scores and Standard Errors are reported as loglts. For a definition 
of logit see Chapter IV. 
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Rellabllty Coefficients 

A second reliability consideration Is that a score on one fonu of the 
examination should be cos^rable to scores on other forms of the exsninatlon. 
Part of this concern is handled by careful calibration of Items and person 
scores so that the various foras are equated. (See Chapter IV. ) 

Another way of lcoMng at rellnblllty Is to consider the reliability 
(^efficients for ram^ collections of items frm the item pool, such as those 
provided by the seven field test form of the exafliination. 

Coefficient alpha for each of the field test fonas of the various sub- 
tests is reported In Table 3.2. These coefficients are remarkably consistent 
across forms and are acceptably high. 



TABLE 3.2 



a^FFiciorr alpha m various forms 

OF THE FTCE FIELD TEST 





No. of 
Itans 


Test Form 


Median 
Alpha 




1 


2 


3 


4 


5 


6 


7 


Readlnn 


50 


.82 


.81 


.81 

* 


.82 


.83 


.77 


.83 


.82 


Math. 


27-2B 


.83 


.84 


.86 


.88 


.82 


.83 


.33 


Prof. Ed. 


69-70 


.81, 


.79 


.79 


.80 


.82 


.82 


.82 


.81 



Because tte field test forms of all subtests »<ere shorter than the actual 
forms of the exsaination will be, the Spearman Brown prophecy formula can be 
used to estimate the reliability for tests of the length of each subtest of the 
actual examination. Estimations of tte total test reliability derived through 
this process are shown in Table 3.3. Again, these values are quite acceptable. 



TA6LE 3.3 



ESTIMATED RELIABILITY FOR LENGTHENED 
SUBTESTS OF FTCE 





Numter 
of Itams 


Proposed 
Length (L) 


1 ! 

^dian 

Ob^ic^^ed. 
Reliabfllty ^ 


Estimated 
Reliability 


Reading 


50 




80 


.82 


.879 


Hath. 


27-28 




40 


.83 


.883 


Prof. Ed. ; 


69-70 




100 


.81 


.859 
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Both corrected and uncorrected reHabllltles met the usual standards for 
educational achieven^t tests. 

Reliability of Scoring of the Writing Subtest 

The major rel iability concern for the Writing test is the inter-judge 
reliability of the ratings. In the study frm n^lch tl» reliability data mre 
obtained, essays were tiritten by 360 teacher education stu<tents at tuo teacher 
universitles.r Each essay addressed 01^ of six topics. The jw^s Mere selected 
and trained by the sane procedures which are being used in the actual achiinis- 
tration of the exoainatlon. Each essay was rated Independently by three judges 
with a referee to reconcile discrepant scores. Two approaches were used to 
estimate the reliability. Each of ttese is described below. 

First » four indexes of inter-rater agr^ioent were owted and compared 
with "target** values (terived fron the literature. These are reported in 
Table 3.4. 



TABLE 3.4 

IHTER-RATER RaiABILITY OF 
HRITir^ SUBTEST OF FTCE 

Raters' Level 

- % Complete Agreeim.nt 32.2 



Index 1 - 
Index 2 - 



- Average % Two of Three 
Raters Agreeing 98. 3 



Index 3 — Average % Agreenent by 

Pairs as to Pass/Fail 81.3 

Index 4 — * CoB^lete Agreement 

about Pass/Fail 71.7 



Target Level 
30-40 

80-90 

80-90 

70-80 



id 



On three of the four indexes the rating team's level of consistency fell 
within the target ranges; in one case. Index 2, the team's level exceeded not 
only the target range, but also the level it had achieved in the training session. 

In addition to the four indexes. Coefficient alpha was coisputed for pairs 
of raters and for the rating team both before and after the substitution of the 
referee's ratings. This coefficient indicates tte expected correlation between 
the ratings of the team on this task and those of a hypothetical team of simi- 
larly comprised and similarly trained raters doing the same t£sk. Table 3.5 
presents the coefficients. 



7The reliabtnty of the scoring of the writing test also was nronito-.^d 
during the scoring process at the University of Florida for each actual test 
administration.-^^The results of these studies are reported elsewhere. 
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CQEmCIBIT Aim FOR Ii lltll4M fflE» ISLUULiniS 

Mem 1 and 2 .010 «OiO 

Raters I and 3 « .720 ' 

taters 2 and 3 ^ * .» .919 

Raters 1, 2, and 3 .759 .flOO 

Tiie f fgwrts In time 3.5 ivflect tiie efflict of i««nr«o*o f«l^ M Ite 
ten's between rater conslttmy. femaifeo tfce lejel «f mlWIIty to fWY,,^ 
Instance wad InereMli^ ft st^tantlalljr 9aBa. m ooeffwtot 
-tint of raters 1, 2, mid 3 (I.e.. tin tinle «om) tOo vel^U ^ 
Is. as Miald to expected. tHe lif^iBrt* ilme tto tvllMllly of • pmp of MiMd 
raters oenenlly Incraasn as Its nmtKr Inmaset md ilnoo «to fiMmtiOB of a 
referaeV rat^^ Is, In md ef Itself, a dilltorate mmr4 i^t^Mit fti Inter- 
rater rellaMlltr. tlie level ef retl^lllty acklevadlr M ftttao tmm aaets 
acoBptable standards for such rating. 

Tlie Flmrlda Teacter Certification ExMlnatlon lias ^sqaite ni1laMI1l;y f» 
the purposes for tihlch It m deslMd. Ito rellMlllafO^ ^ m'^le ctolee 
tests WIS calailated frm data darfiwd frm the /Mwrll 1900 fteld ^* RsllaMlltjr 
ooefflc1«its Mre expressed In tenes ef stwdn^ errort caoffloloiit elpHa. end^ 
Spearm-Oroisi Fro|rt»qr values. Tto Ust reliability ft fMimr Mftand eack 
tiee the test Is adelnletered. hut thise Me are rapmted el i i w to i^ , « 

For the erltlno test, reliability of raters' Judu i mti una smltmO In a 
special pilot test of the prooadam. Hie raters* dacfiloi» dttt^ao Ito seoHnf 
of the actual tests Is smltorml by thi test support omitractdr. tto tol varsity 
of Florida, and Is r^rted elsewhera. 
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CHAPTER IV 

: %am scaliis m passing $cms 

In order to set i^iising scores m the sultiple ctolce subtests of the FTCE, 
there Mere tHo najor. tasks to be accwilshed* First. It ms necessaii^ to 
deteriilm the i^oraance level on each »ibtest which tmild be Judged satis- 
factory and silcondly. It ms necessary to devel^ a system to «9iftte i^rfor- 
mance across different ranss of the exaaination. Althoush these are separate 
tasks. It Mas considered desirable that the outcome of both be Incorporated In 
a single rule; that Is, khd rule for passing scores on the exasHnatlon nould 
be expressed in terms of a scaled score idilch wuld be used to equate various 
forms of the exanlnatlmu 

Score Scaling 

The isatter of the snled score was ad<b?8ssed first. Very early In tt» 
development of ttm exasiljnatlon It «fas ^1<ted to use a non-miraiatlve scale* and 
the Rasch s»del was selected as the scalli^ and ei^tli^ oethod to be used.i> 

The Rasch noitel is one of a f^lly of neasuresent ao^ls develi^ by 
Gebrg Rasch. It Is being wlitely used In 1ar^«-scale testing prograiss. Both 
Item difficulty and pe^rson ability are calibrated on the sm scale. Person 
ability is expressed as a fimctlon of the difficulty level of Itens idilch an 
examinee can anstier correctly and IteRi dlffloilty Is expi^ssed as a function of . 
the ability of examinees «^ answer the Itai correctly. 

Both person ability and itm difficulty are expressed as a logit, a term 
coined frora "logarltlw odds transformatlwi, ■ The logit for a person's ability is 
the natural log odds for succeeding on Itm selected to define the "zero poinf" 
• (.00) on the scale. Ihe logit for an Iten's difficulty Is tSm natural log odds 
for eliciting failure frow persons at the "zero point" (.00) m the scale.* 

Loglts range fron approxinatety -6.00 thriHigh .00 to 46.(K>, with scales for 
inost tests r«iging from -3 to ^3. The logit scale can be us^d to interpret item 
difficulty and person ability. Noi«Bver, the ImH scale has tm drawbacks. First, 
it is t decimal scale, and second. It has imgatlve numbers. In order to make the 
scale more i»mvenlent to use, it shffiild be transforaed to cover a different range 
of values. In making the transformation, several properties for the mn scale 
wert iiqK>rtant. First, tte transfotnatlfm shcnild be mathematically sli^le. Second, 
the transformed scale st»uld preserve all the Information In the li^lt scale. 
Third, the scale shcjld not be easily confused with other scales In cosnsn usage. 



Bwinutes of the^^wncil on Teacher Education, January W9, 

9for additional! Information about the Rasch model, see references 
in Appendix G. 
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In order to determine an appropriate scale transformation, three questions 
had to be addressed. First, what constant muld bejised as a imiltlpll cation 
factor? Second, what additive constant would be used to eliminate negative 
numbers? Th1rd,.where should the scale be anchored? Answers to these questions 
would establish the characteristics of the new score scale. 

The multiplication factor was selectedcto be 20. This vrould eliminate any 
decimal values for scores and make It possible to round whole numbers without 
losing any Information. Thus, no two adjacent raw scores would have the same 
scaled scores. 

• 

The additive constant was selected to be 200. This would permit the scale 
to be expressed entirely with three digits and would not generate a scale Identical 
to any other cainronly used scale. 

last. It was determined that the scale would be anctored at the passing 
score for each subtest; so, even though the subtests vary In difficulty, the 
person performance level nec&ssa.ry to pass each subtest can be made Ictentlcal. 
The passing score valuers set at 200. 



In nreparing for the task of setting the standard for passing the various 
subtests of the Florida Teacher Certification Examination, an extensive review 
of the literature was undertaken. Fortunately, many of the most relevant papers 
have been collected In three publications, and earlier papers are jclted In these 
publications, so that the task of Identifying the state of the art was greatly 
simplified. 

The first of these publications is the Florida Journal of Educational "Re- 
search , Volume 18, 1976. This journal published papers presented In a symposium 
at the annual meeting of the Itetlonal Council on Measurement in Education (WME) 
in April, 1976. The second publication Is the journal of Educational Measurement , 
Volume 15, No. 4, Winter 1978, published by NCHE, which was a special Issue on 
standard setting. Finally, NCME Issued a monograph. Practices and Problems 
In Competency-Based Heasuranent , 1979, which was prepared by a task force as the 
culmination of three years of study and symposia. Many of the papers in this 
monograph also appear in the other publications. 

In developing the procedures to be used in setting passing scores for the 
FTCE, the materials cited above were very helpful. However, the procedures 
rec(»niiended for the Florida Teacher Certification Examination are not directly 
adopted from any of the published material. As Bunda and Sanders state In their 
preface to the standards chapter of the 1979 NCME monograph, "The reader will not 
find a little jiffy formula in this section which will allow for the setting of 
standards. The choice of a standard is a complex decision." In his pa^er in the 
same monograph, Conaway states, . 



"A review of the literature shows that a definitive set of 
procedures for setting standards in various types of competency- 
based programs dofes not exist... it is apparent that practitioners 
cannot obtain validated standard-setting procedures, either off- 
the-shelf or out of the literature. Instead, they must devise 
their own standard-setting procedures, relying upon their own 
experiences and philosophies in cmipetency-based measurement and 
upon any guidance they can get from available literature and 
fellow practitioners." . 



Setting Passing Score Standards 




This Is idwt the Departmt of Eihicatlon did fior the FTCE. The procedures 
are described in the folloMlng sectlms of this chapter. 

Several procedures for setting stsidards have been used or proposed In the 
literature. They generally fall Into one, of^fcree cate^rles. Each of tfwse 
categories Is presented and discussed In thffjbllowing pages. 

Requirement for Perfection 

The* "requirements fw.- perfection" B«tl»d sl^ly selects sone passing, score 
less than so that an allomce Is isade for "careless errors" or "lapses 'comiion 
to all people." Advocates of this procedure often Imore, or are wiaiare of. the 
fact that Items measuring a given objective or subskill w vdiy greatly In diffi- 
culty because of such Irrelevant factors as ambiguity In Mording and the clol6ness 
of the d1stl*actors to the correct (keyed) answer. 

The Use of Noraatlve Data * 

Standards can be set through the use of any of three types of noroatlve 
data. First, one can sl^ly look at the distrlbutlim of actual perfonsance of 
the group of applicants and set the passing score at some point on that distri- 
bution. Mien only a small percentage Is to be selected from a large ntssber of 
applicants, (as In selecting candidates for medical sctool, for example) this 
method works quite i<ell. It Is not likely toKork so well In selecting trnly the 
"incon^etent." 

A second way of using normative data would be to (teteralne or estimate i^t 
pi!tfportion of beginning teacters presently employed are "Incompetent" on the 
skills measured by the examination and set t»» cutting score for the first year 
In such a way that a specific percental of applicants Is failed. In subsequent 
years, the cutting score would remain the sane, even tlKMigh fewer and fewer appll- " 
cants might fall because of Improved selectlmi and training In the teacher train- 
ing .Institutions. 

A third way of using nonnatlve data would be to determine the number of new 
teachers needed each year and set the cut-off score so that a sufficient number 
of applicants "pass'' to provide for the needs. 

None of these procedures seemed acceptable as the basis for setting the pass- 
ing score. 

Opinion of Judges 

The third method Is known as the "jury mettod" and several different 
procedures using "judges" have been devel(»ed. The judges are usually selected 
to ifepresent some or all of tte Important ^clienteles" of the test. Host of the 
procedures described In the 11teratuif« were (teveloped for use with course exam- 
inations and tte "jury" was ma(te up of Instructors In the courses. 

* I ^11 of these procedures require that the judges conceptualize a minimally 
oV- marginally qualified person and keep that person in mind while making their 
judgements. However, the operational definitions of those terms vary greatly from 
one situation to aiwther. 
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Tlw procedure <teveloped fay n&to"^^ (19S4) requires judges to eliminate for 
each Item those optlmis idilch the *liiWBSt student could eliminate. The proba- 
bility of that stwtent ansi«r1ng the Item correctly Is the reciprocal of the 
number of the opt1<ms left* Tl» sian of the reciprocals Is the "guess score" for 
' the "loMest D" student* and ttm passing, sa)re Is set scwenhere above that point. 

. Ebel 's (1971) proce<hire requires Judaes to rate each 1t«i as to relevance 
and difficulty fbr a *w1iiimally qualified (barely passing) applicant." Then a • 
table Is used to assign a "probability" value to each 1t«n based on tte a»ib1na- 
tion of the ratings on relevance and difficulty. Finally, the avera^ Item prob- 
ability Is deteiwlned and that becomes the cutting score. 

Angoff's (1971) procechire sii^ly requires that each judge state the prob- 
ability that the "minimally aci»ptable person" irtll answr each Item correctly. 
The SUB of the probabilities Is tte cutting score. 

All of the preceeding pr^edures require that judges estimate the difficulty 
of items. If judges (k> not have expertise In ttm area, tlwlr estlnates may be 
grossly Inaccurate. Furttermore, it seems ridiculous to ask judges to estimate 
1t«n difficulty when empirical data are available, as they were for the FTCE,, based 
on the field test. 

In a different kind of procedure, ju(^es are not required to Judge Item 
difficulty. Instead, th^ are proy1<ted smples of calibrated Items, arranged In 
order of difficulty, from easy to hard. They are asted to litentify the dividing 
line between itrais which the minimally c^allfled examinee should generally answer 
correctly (probability greater than 501) and those for which the probability Is 
and should be lower (Draba. 1979). 

It was the last procedure that was used for the FTCE. The jury used was the 
Council on Teacher Education (COTE) which met in Wakulla Springs June 24-26, 1980, 
and developed reccawendations for standards for passing each subtest of the FTCE. 

Procedures Used for the Florida Teacher Certification ExwiinationlO 

The judges who recM^ded tte passing score standards for each of the 
subtests of the FTCE were current and fornsr members of /the Council on Teacher 
Education (COTE). COTE had developed the co^tency-baied w»del for Teacher 
Certification, conducted the studies lAlch 1<tent1f1ed these cfflnpfetencies, and 
validated subskills and specifications for the examination, so It was very fitting 
that this group should determine tte. performance, level to be required In order to 
pass the examination. There were twenty-one jydges whaassanbled In Hakulla 
Springs on June 24, 1980, to beain the process of setting the passing scores. 
(See Appendix A for the roster.) ^ 

Before the meeting, participants received two docunents for study. The 
first document was the history of the develo}W«nt of th^ exwiination. The second 
document provided technical infonnation to be considered in setting passing scores. 
The latter document covered the following topics: Over|1ew of methods used in 
setting standards* Issues in setting standards, and,/eciiwendafJons for score scaling 



lOpor a complete description of these procedures see COTE Minutes, 
June 24-26, 1980. " 
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The procedures to be used In setting passing scores were developed In a 
conference beti«eefl Or. Garfield Wilson. Aibslnlstrator of Teacher Certification, 
Dr. Dale^ke, Behavioral Science C^sultant; and^ Dr. Annie Mard. Measurenent 
Consultant. Dr. lake agreed to TeaH total group discussions and to .facilitate 
group process In onter to help the jud^ arrive at decisions. 1^. Ward pro- 
vided technical Infonaatlon In both inrltten materials and oral presentations, 
an(H>r. Wilson supervised all aitelnl strati ve details. Including Invitations to 
judges and other Interested people. 

At the beginning of the Wakulla brings neeting. Dr. Lake discussed the 
procechires to be used and Dr. Ward discussed the teclmlcal Infomatlon distrib- 
uted to participants. Staff recomendatlons for passing scores were given to the 
participants. It was agreed by the total group that Its deliberations wwild 
result In recomoended passing scores which would balance teclmlcal Issues, 
social end political concerns » and practical considerations* ■ 



In subseqi^t sessions* the various subtests were conslitered one at a tine 
until consensus was reached about the recomendatlons for each subtest. For each 
subtest the procedures were as outlli^ below. 



1. Tn a large gnmp sesslim. Dr. Ward presJit^ the staff r^n- 
mdatlon and the rationale for the reaamendation. The 
rationale covered such points as: 

a. General nature of Items on tim subtest* 

b. Level and range of difficulty of Itm on the test, 

c. Characteristics of the distribution of soires for^ 
the test (I.e., amount and nature of skewness, abrupt 
changes In slope or breaks In the distribution);. 

d. lB9>act data; and 

e. Research Infomatlon If available. 

2. Judges asseipbled In nixed role groups. They were given samples 
of Items for the subtest, selected from a 1^11 ran^ of diffi- 
culty levels. Legit values fdr each Item were pmlited. 
Instructions ref^ilred that the judges sort the Items Into two 
groups: (a) those which applicants should be expected to answer 
correctly more than half, and (b) those iriilch aoollcants should 
be expected to answer correctly less than half.*' 

Then, the judges were to determine the dividing point (In loglts) 
between the two groups. Judges worked Inctepenctently at first, 
then atten^ted to arrive at group consensus. Finally, they 
coiupared trmir decision with the staff rea»nendat1on, and either 
accepted, modified, or rejected that recommendation. 

3. As soon as all groups were rea^y to report, they assembled in a 
group of the i^le. At this session, each gmp leader presented 
the recommendation for the group and noted praMems or concerns 
of the group. All these reports were- recorded and displayed. 



, ^See Appendix D for Agenda and Appendix F for materials. 

12see COTE Minutes, June 34-26, 1980, for a more complete description of the 
procedures. 
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4. Jifdf^s reassea4>1ed Into role-alike oroups. In these groups they 
considered political and practical l^llcations of the recoiieien- 
datlons. Itomtlve data for various ethnic qrw^s and programs 
Wiere provlited fbr consideration. 

5. After tte snail group deliberations Mere oni^leted. tte total 
group was reassaiAled^ 

m 

IhR original plan called for the final recmmidatlon for each'^suHest to be 
made at this |H)1nt. IMw. *rtth the very first subtest, concern nas e)q)ressed 
about setting passing scores cm the varlmis subtests In a piecemeal fashion, so It 
Mas decider to itevel^ only t^ta|1ve reomndatlons at first, ti^ to reconsliter 
all recosna)<tet1(»is as a package at the end of the loeetlf^. Tl^refore, tte pro- 
lagc t wei ^ h - g^t^l mminq m% a tentative recoswendatlon and a list of ooncems 
to be re-exaniiKM later. 

The proce<hire for the Vrltli^ subtest necessarily differed sosen^t fros 
that ftescrlbed above for the oultlple-chol^ tests. In the first place, the am- 
tractors Mho had develoj^ tte Writing ai^test had develi^ a scoring systen which 
Incorporated a sUndard of •acceptable*' or "wiacc^ble" j^rfonsam*. The 
judges* task ms to exmilne the plication of tl»t standard* Samples of essays 
Mere provided to the Jutes, alofw with the ratings fron tinree trained raters 
and. In siaie'cases, a referee. Tm Jud^s were asked to consliter whether they 
felt the essays had been correctly classified, particularly those which were 
judged either "Unacceptable" or "mnliaally Acceptable." . , 

itecosnendatlons 

After all deliberations, the judges reconsended acceptance ©f the staff 
reconnendatlons of all subtests except the Mrltlng subtest, for which they recom- 
mended a slightly hl^ier standard. 

The recomsended passli^ scores (esipressed In loglts) are listed below, with 
a note about the concerns and considerations ndjlch were expressed about each, 

Readlnfli Perfonsance equal to a logit value of 1.40. 

Concerns ; 1) This standard Is above the 75% cdrrect level 

reconaencted In research. 

2) A high rate of failure for Blacks and some 
program areas my result. 

Consideration ; Items are very easy, and there were many 

perfect scores 
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Mathematics ; (>erfoniance ei^ual to loglt valtffi of i.(K) 



A hlgH rale of failures, particularly for 
Black stiKtents and soae progran groups, 
possibly will result. 



Considerations; I) 



2) 



Items are not too differsit fron high 
sctool State Student Assesmit Test. 
Prospective tethers shmild be expected 
to ito as iiell ais high school stuitents. 

Nathmtlcs Is n easily trained sMll, 
so those tfho score low c«i liprove 
their perforaance In a stort i^rlod of 
tise. 



Professional Education ; Performance equal to loglt vali« of .25 

Concerns: 



1) 
2) 



Considerations; 1) 



2) 



Mmy are Itens possibly Irrelevant for 
the Voc/Tech grotq>. 

I 

There Is a possibility that the scores ' 
R^y be "over-Interpreted;" perfomance 
assessment In the Professional Education 
area Is 1s|M>rtant. 

Items have been judged directly relevant 
to Identified competencies, subsMlls, 
and Sf»c1f1cat1<ms. 

Itens have been developed to achieve a 
balme across grade levels and subject 
matter. 



Writing ; A total score of six or more based on the summed rating 
of three trained Judges lelng a scale of/iSfie to four; 
at least tiw of three judges must agree en the accept- 
ablll^ of the writing ssqile. 



Concerns : 



1) 
2) 



Co nsiderations ; 1) 



A high failure rate may occur. 

It may be difficult fbr applicants to 
Improve their performance In a -short 
period of time. 

Even the "minimally acceptable" essays 
are of poor quality. 



These reci^nendatlons were suMtted to the State Bo4rd of E^catlon as a 
proposed Administrative Rule. The Board adopted the recoBnendatlons as Rule on 
July 30, 1980. 
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CHAPTER y 



BIAS ANALYSES 



Test bias has teen iteflned In variims ways, (bie set of ^Inltlons relates 
to the way tests are used and the equity of <tec1s1ons that are based on the tests. 
Another set of definitions Is related to tte contratt or the foraat of the Iteas on 
the test. Bias also Is a validity Issue because » to the extent that scores on a 
test reflect ^rw^ 8»i^rsh1p ratner than the ImoMlei^ or skill the test Is sup- 
l>osed to msure, the test Is Invalid. As Sh^rd. Caollll* ffitd Averlll (1980) 
say, "a test Is biased If equally able Individuals, froai dlffieroit grot^s, do not 
have equal probabilities of success." 

Note* hoMever. that differential perfonamce of different groups Is not 
adequate evidence of bias. An exa^»1e f^ another field will Illustrate this. 
If a group of undernourished children and a group of Mall-fed children are lelghed, 
and the well-fed children are found to have higher weights than the undernourished 

?roup. It cannot te concluded ttet the scales are biased. The differential weight 
I.e., "performance**} reflects acoirately the difference In the two grot4>s as to 
nutrition. 

Several nethods of Inwstlgating bias have been dei^l(H>od and reported In 
the literature. Niny of these proce<^res are related to caclslwis iiade on the 
basis of the scores.^ The potential bias of decisions stade on the basis of the FTCE 
will necessarily have to te delayed until criterion data are available (e.g., 
perfonoance rating on teaching). However, care has been taken to ensure that the 
content of the test Is not biased. This has been accoi^llshed In two steps: 

1. In the Item developaent sta^» a series of reviews wai used to 
screen Itenis for apparent or potential bias and fbr offensive 
materials. 

2. Data fron the field test were analyzed statistically to 
Identify Iteas idilch perfOnaed differently for different 
groups. The Itei^ flamd by statistical analysis were 
then Inspects for possible biasing elmnts. The data, 
were analyzed for sex groups, ethnic grot^>s, and 
progrffiD groups. 

Qevelopnental Procedures 

The procedures used In <tevelop1f^ and reviewing test Items for potential bias 
Incorporated tte practices laost fr^iuently recoraaent^ In the literature. A set^ of 
guidelines for avoiding both ti» appearance\of bias and biasing elements was pro- 
vided to contractors and Incorporated Into revi«« form. (See Appendix E.l.) 

Each contractor who wrote Itens for the FTCE was required to have all items 
reviewed by at least three experts in tte area, and at least one expert ms re- 
quired to represent a ninority ethnic group. Review teams were also required to 
include both men and wooen. . In addition, contractors n^re required to administer 
each Iten on a one-on-one basis to at least three students, one of whon was re- 
quired to te a minority person. Both sexes and a variety of program areas had 
to be represented. (See Appendix E.2 for tl« review forw.) 
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After all Itans Mere written they were pilot tested and then reviewed by 
expert revieiiers for potential bias. T)» bias review team was mfte up of five 
members, consisting of two Hispanic and thr^ Black members. Two of the teem 
were male and three were female. The bias review was conducted by Or. Tom D. ^ 
Freljo. a measurement expert with considerable experience In item review procedures. 

Two Reading passages were flagged as having "sexist" language (I.e., use of 
generic 'he'); one Mathenmtlcs item and one Professional Education item were con- 
sidered as offensive to one or more ethnic groups. All of these Items were 
replaced. 



Statistical methods l»ve been (teveloped to Identify items on which groups 
perform better or worse than their overall performance on the test would predict. 
The flagged items are then inspected to ascertain whetter there are biasing 
elements In the item. A review of the literature on statistical procedures for 
detecting potential Item bias has been yreatly facilitated recently by a paper 
presented at American Educational Research Association (AERA) by Shepard, C^nilH, 
and Averin (1980). This paper cites several very con^SF^^slve reviews of Item- 
bias detection methods and reports the results of a study In which six of the 
best known approaches, soii« with more than one variation, were con^ared. 

The data used la the FTCE-studies of potential item bias were collected In 
the field test In April 1980. The methods for analysis of bias of Items on the 
FTCE were: 



1. the transformed Item OifficuUy Method (Angoff) Introduced in 
Angoff and Ford (1973); 

2. the One-Parameter Item Characteristic Curve Method (Rasch) 
(Wright, Mead, and Draba, 1976); and 

3. correlational studies. 



These were the only methods for which sufficient data were available, because 
although 1,186 people participated In the field test, items were distributed across 
seven test forms, so M's for some test forms were as small as 150; When these were 
further sorted into etimic or progrMi groups, the N's became very minute in some 
cases. Sonw groups were combined, n^n the combination was a logical entity. In 
order to fom groups large enough to analyze. 

Before comparisons were made, items with values lower than .05 and higher 
than .95 were deleted because for these Items group differences are almost certain 
to be simply chance differences. In addition, itans deleted from the pool for 
technical reasons (poor discrimination or nwdel fit) *«re not considered. The 
nwnber of items deleted for these reasons Is Indicated below: 



Statistical Procedures 



No. of Items 



pi .05 or 
i.95 



Technical 
Deletions 



Remaining 



Reading 
^thematics 

Professional Education 



230 
102 
369 



141 
5 
6 



4 

29 



89 
93 
334 
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TIP COTparlsons (Angoff) 

The Transfonncd Itai Difficulty (TID) approach relics on an Item-by-group 
interaction definition of bias. Items are flagged for further bias review If 
they are relatively !i»re difficult for one group than for another. Angoff and 
Ford (1973) IntroAicfcd the itm-dlfflculty delta plot method. It Is one of the 
most conmnm n^thods In practice and Is routinely used by the Educatlml Testing 
Service as a screening device In test devel(H»iient. Item difficulties or p-values 
(the proportion of examinees ^ttlng the item right) are coi^ted separately for 
each group, then each p-val«e Is trahsformed to a z-value correspcwidlng to the 
(I - p)t:h percentile of the standardized normal distribution. The prliMtpal axis 
line Is then computed. Iteas wMch ctevlate most from this line (neasuild as a 
perpendicular distance) are flagged as potentially biased. The direction of the 
difference Indicates the group for which the Item Is relatively easier. 

The following cosparlsons were made for all Items using the /togoff Method: 

Sex group 
Male to fm&ie — 

Ethnic Group 
Black to l^lb 
Wilte to Hispanic 

Program Groups' 
Secondary Progra^ to All Other Programs 
Elementary Programs to All Other Progrwis 
Vocational Programs to All Other Programs 
Special Education Programs to All Other Progrims 

An Item was flagged for review If t»« "standardized Mas'* value w^s ±2.00 ftr 
greater, indicating that the delta difference for the two groups \ms twice the 
standard error. 

Of the 89 Reading Items conslftered. 18 i«re flagged for further in-itepth 
review. Performance on four of these favored Secondary E<hicat1(Hi majors, and 
performance on five others favored Special Education sijors. An Inspection of 
these Items does not provlcte any obvious reason tlwse Items should be 
relatively easier for these groups. For the other flagged Items there was no 
consistent pattern. 

Thirteen of the 93 Mathematics items were flagged for inspection. 
There was a tenctency for the flagged Items to favor males (3 items). Secondary 
Education majors (4 it«ns), and Vocational Echication majors (4 items). Inspec- 
tion of the flagged items did not reveal any biasing elements, except that one 
which favors Vocational Education majors has a vocational class setting; 
howeveri items with elementary school settings did not favor elementary majors. 
The other flagged items involve simple con^nitation of fractions or decimals. 
It seems unreasonable to think that the items are really biased toward or against 
any group. 
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Twer^ty-one of the Professional Education items were flagged for, review. Five 
items favored Secondary Education students. These were all related to either 
Conpetency 6 (Personal /Social /Academic Oevelopaent) or Cof!^)etency 14 (Diagnosis 
and Assessment). Four Items favored Special Education majors One of these 
was related to Competency 6, the others were related to Conpetencies 11 and IZ 
(Teaching Methodology). Three of the four items which favored Vocational Educa- 
tion majors were from the Teaching Methottology area and the fourth was related to 
student assessment. Five items favored Hispanlcs, two favored Whites, and one 
favored Blacks. There was no simple explanation for these results, s.nce the 
items cut across all content areas. Furthermore, ttere was no feature of the 
context which seemed to be related to any ethnic group. The settino for the 
items covered five different subject areas, and there was no specification of sex 
or ethnic variables in any of the items. 

In sunmary, a few of the flagged items favor certain program groups (Secondary, 
Special, and Vocational Education), but none are biased against any ethnic or sex 
^ groups. The greatest percen tage of flagged items were Mat hematics Hcts and involved 
very basic mathematical procedures (e.g-.-addltion, suDtracTToft , anT"d1v1s onjot 
fractions; percent; ratio; etc.). The Reading and Prefesslonal Education Hems which 
favored Special Education, Secondary, and Vocational Education students tended to 
come from the Educational Psychology field. Rather than reflecting bias, it 1s prob- 
able that these results reflect current curricular emphasis for these program areas. 

Rasch Analyses 

Rasch model n«thods for identifying biased items are based on the concept that 
examinees with the same ability should have the same probability of getting an Item 
correct regardless of group membership. After difficulty parameters are equated to 
?he same s?ale. the moit straightforward index of bias Is a simple difference in the 
difficulty parameters estimated separately for two groups. 

The Rasch analysis for item bias was limited to linking items, e.g., the sub- 
sampled items which Lre comnxjn to all forms of tt^ field test. These coinnon terns 
had been selected to represent the pool of items for each of the subtests. Using 
the connon items made it possible to make comparisons for groups which were too 
small for intra-form comparison. The nwnber of items involved was as follows: 

Reading: 2 passages, 20 items 
Mathematics: 15 items 
Professional Education: 20 Items 

The following comparisons were made using the Rasch Model: 

Sex Group 
Male to Female 

Ethnic Group 
Black to WhUe 
Hispanic to White 
Black to Hispanic 

Pro gram Groups 
El ementary to Second ary 
Elementary to Vocational 
Elementary to Special Education 
Secondary to Special Education 
Secondary to Vocational 
Special Education to Vocational 
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An item was flagged as potentially biased If the differences In difficulty 
loglts exceeded ±2 standard deviations. A simnary of the results follows. 

leading Subtest Assessing bias for Itens In the Reading subtest was co^llcated by 
the extreme easiness of the test. Of the 20 Itesis irtilch were used as links, only 
three items (2, 13, and 18) had p values below .9S, and one of those (13) had a p 
value of .90. Therefore, It was declited to limit consideration of bias for only 
those three Items. ' 

Only one of the 30 o»0ar1sons (3 Itens x 10 coRipaHsons) was flagged, less 
than the nimber to be expected by chance alone. Inspectltm of the Item and the 
passage In which It was linbedded did not reveal anty obvious biasing el&nent. 

Wathenatlcs Subtest None of the 15 linking Items for Mathematics was eliminated 
because the p value was greater than .95, so the group conparlsons were maite on 15 
items, making a total of 150 C0B^ar1s<»»s. Seven of these differences were statisti- 
cally significant at the .OS level, exactly the number which would have been expected 
by chance, Furthermore, the direction of the differences varied across all groups 
, with one each favoring males, white, black, Secmidary, Elementary, Special Education, 
and Vocational. 

Professional Education Subtest Of the 20 common Items for Professional Educa- 
tion, two were renoved from this item pool for technical reaswis; none were ellminatf^ 
because of extreme easiness. Therefore, there }nere eighteen (18) Itess Involved In 
the comparison, providing 180 co^rlsons. (^ly six ($) of these differences were 
significant at the .05 level, less than the niwiber expected by chance. These were 
distributed in favor of Hispanlcs (2), Special Education, Secondary (2), and 
ElenMentary. 

In summary, the Rasch analysis did not reveal any items which could be con- 
finned to be biased. 

Correlational Analyses 



Correlations were congnited for the Rasch Item difficulties for each pair 
of groups for the Mathematics and Professional Echicatlon^ subtests. The Reading 
subtest was not Included in this analysis because of the small nimber of items 
involved. 

The correlations are reported in Table 5.1. For Mathematics, the correlations 
range from .710 (Special Education-Vocational Education) to .914 (Elementary- 
Secondary) with a median of .864. For Professional Education the correlations range^ 
from .887 (Elementary-Secondary) to .976 (Male-Female) with a median of .930. 

These correlations indicate that the Item difficulties for the Professional 
Education^ subtest are generally in the same order for all groups. For Mathematics, 
there Is slightly more variation between groups, with both of the lowest correla- 
tions Involving Special Education majors. 

Sumroaty 

The Florida Teacher Certification Examination was extensively reviewed for 
cultural, sexual, and racial or ethnic bias. The bias review procedures included 
statistical procedures and reviews ef the Items by professional educators. The 
final set of approved items are free of bias. 



TABLE 5,1 
C0RRELATI<»4S OF ITEM DIFFICULTIES 

Btimn VARIOUS groups 



Wathefflatlcs Gvom 



Kale - Femile 
Black - mu 
Hispanic • mu 
Black - Hispanic 

Elenwjtaiy - Secondary Education 
El«aentary - Itocatlonal Education 
Eloaentary - Special Education 
Secwidary - Special Eihicatlon 
Secondaiy - Vocational Educatlim 
Special Education - Vocational Education 



Professional Educatlwi Groups 



Male - HmU 

Black - White 

Hispanic - lihlte • 

Black - Hispanic 

Elementary - Secondary Education 

Eleaentary - Vocational Education 

Elewentary - Special Education 

Secondary - Special Education 

Secondary - Vocational Education 

Special Education - Vocational Education 
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A. Professional Contributions^ 

Many people Haire participate In the toeliHimt of the Horlda Teacher Certi 
ffcatlon Exaiilhatlcn. Sfloe of «iese nlll i««ain anonyaous Uja., those >iho 
participated tfie pflot test or field testing)- Those tfltlidlrect respmisl 
bllltles are /listed beloM: 

Director: 

_ Dr. fiarfield Ullson* tMrectorof Teacher Education . 
Departaoit of Educatlcm 

Measurement Consultants: 

ftr. Jao^ Beard, nortda State Uhlverslty 
or. Aimle «ard. university of South Rortda 
Dr. Howard Stmr, Florida State Uhlverslty 
rte*. Robert Felnberg, ttalwrslty of Florida 

Staff Assistants: 

Wayne Hatrls (1978-79) « 
Chrtstlan Doolln (1979-W) 

COBiwtency Sunriy: ^ ^ ^ 

wnilasi Davles, Director of Federal Programs and Grants, 

Polk County Sdmols 

Preparation of Specif 1cat1oi» Bulletin and Technical Itanual ; 

Dr. Annie Hard* Pfeasurasent and Research, Ihilverslty of 
South Florida 

Reviewers of Tedmlcal flanual: ^ 

^ Dr. Robert Ebel, Hlchlow State University 
Dr. wnilam Itehrens, Michigan State Itoilversi^ 
Dr. Roger Mllfc, University of South Florida 
Dr. Howard Stoker, Rorlda State University 
Dr. Jacob Beard, Florida State Ibilversljty 
Dr. Thosas Fisher, Departs»nt of Education 
Dr. Garfield Wilson, DepartsiBnt of Education 
Dr. Kenneth Loewe. Departsmfit of Education 
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COTE He^rs (1978-80) 

Classroow Teadiet^ 

Ullllim Itoore, Sealmile bounty 
Quintllla Umdi* Colui^la 'Coimty 
Ter^a Mentimi* Oaite Cmsity 
Uutie nirray, Ouval Cotaity 
Aima (tones* Ouval Cotmty 
Ida Ke1ii^« Brevard Coimty 
Mary i3, ThoBpsoQ, 3arasota Coimty 
NarJoHe HeM. Bronfard Ccninty 
Fannie Nil Haas* Sronard County 
Walter itohnson* Brevard Ctmty 
Tiioi»sena Undsigr* Oade County 

School Atotnistrators ^ 

Rsy Potto rf, SupeHntentat, Lee County 

Dr. Donald Gil! » Sui>er1ntendent> \to1us1a County 

Jms Reese» Pr1i»f|iaU Esc^l a County 

•tolHi Uier, Director of Staff (tevelopi^tt H111st>orough County 

Loimle Coleaan. Principal* Oade County 

I del la Ndss, Director of Teacher Education Center, Sarasota County 
Jeny Sisk, Principal, Baker County 

Sanuel Hunter, Asslsti^t SupeHntendafit, Personnel, St. Lucie County 
Dr. Cedl Carlton, Or^ Superintendent, T^lor Counl^^ 

Higher EducatlOT , ' 

. Dr. Robert Wiepan, Florlcte Atlantic University 
Dr. Nprlan Ca^nl^* Florida Southern College 
«0r. Willis ItolcoBt^* Broward Cmmty Cmimlty College 
Dr. Harcia Mann, ttilverslty of South Florida 
Dr. Eva Wanton, Florida Aji M University 
Dr. Paul Seor^, Unlversl^of Florida 
Dr. Jffioes Rld^rg* Valei^cla Coaiiumlty College 
Dr. Barney Stoutaolre, IftiTverslty of West Florida 

m 

Citizens 

Beebe Milte, Volusia County 
Geraldlne Kalvlnt Collier County 
John Espey. Pinellas County 
Ethel Beckham, Dade County 
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Hoc Task Force 

Or. Jacob'^B^rd. Profasor, Jteasurewoit and Rwearch, 

Flofiito State IMivei»s1ty u s,*^ ^ 

Or. MiVliaa E. Blank, Assistant Pr^S^l^i Vocatiwial 

Education, University of Soutn noHda . 
William Oavies, Director, Federal Programs. Polk County 
linda Denmark, Teacher, Polk County 
Tosi Fackender, Teadier, Leon tounty 
Lewis Jones, I^Mcher, Leon Coimty 
«ary Oohnson.creacher, Polk County ' 
Quintilla lynch. Teacter, Ktolusbia County 
Mary Neirton, Teacher, Polk County 
Susan flor<teng. Teacher. B^ Cojmty^ , ^ -^a^^ 
Or. Daniel Purdwi. Professor, Curricula, University 

of South Horida - 

Dr. Annie Ward, Professor, Measurewent and Research, 

University of South Florida 

Developers of Item Specifications 

Reading: Project "iJ^f ^=,.°SihS-Hnis:"?;™?"l^h-B™w». Or. Carolyn 

Schluck, Dr. Uwrence Hafner «i 
Rev1e*«rs: I^. William Castine. FAMU; Dr. Al Lowe. USF 



Writing: 



Mflthematics: 



Project Directors: Dr. tordon prossell and Dr. Jai^ Hoetker 

Project Staff: Dr. tucy Haywajd 

Reviewers: Dr. Nancy McGee, UCF; Dr. Dan Kelly, UF 

Project Director; Or. Tom Jf"^'*'^* '^Ji -.^^ n.^^ 
Project Staff: Dr. Janice nake. Dr. Walter Dick 



Reviewers: 



Roy Bui doc, UF 
Doyle Casteel, UF 
Arnold Cheyney. \M 
Barry Greenberg, FIU 
Sandra K. Kirk, UNF 
Ronald LaFaro, BC 
Lennie Middleton, 
H. E. Stelner, USF 
Itobirt Tests. FIU 
John Bullock, USF 
M. H. Castine, FAHU 
Gordon Eacte, UWF 
Susan Homan. l^F 
R. E. Lee, lft<F 



Daryle May, JU 

Seaton Smith, \Mf 

Barbara StephenscsiyJU 

Nick Vigilante, FIO 

Ernie Bun^ss 

Richard Bumette, Ha. So, 

Paul Fleming, Fla. So. 

Mike Hynes, UCF 

Shirley Lee, BCC 

Mary Mercer, FAMU 

Judith Staver, JU 

Ann Taylor, BC 

Allen Wuertz. Fla. So. 

j\rlene Brett, IW 
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Professlonal Ei^icatlon 

Personal. Social, Acadqalc 



Project 01 rotors: Dr. Sary Peterson and Dr. Stev^ Boltln. FSU 
Project Staff: 1^. Dan Kelly* FSU; Dr. Stevoi itollln* FSU; 

Peter LlnicMay; ^ne McSrlde; tt1chi»l KoraanlcM; 
Kevin Kurphy; HDtMsn Ccmners: Ill«f Wmi 
dohn Rushbrook 



RevleMers: 



Technical Skills ; 
Project* Directors: 



Dr. l^nlel Runkm, USf; fSr. Seoul Arreola* BU; 
nr, Mertert A1$x«Kter, fmii Sara Butzln. Nanr 
Calhoun* Brenda \Co1tfBbo> JaiBK Pitts, (all 
teadiers» FSU DmlojHBental Itesearch SdK»l) 

Dr. Anna (telson and Dr. Millard Nelson, FAU 
Dr. Charles Iteiuban, USF; Dr. Annie »ard, USF 



AdBlnlstratlye Skills; 



Project Director: 
Project Staff: 
Reviewers: 



Developers of Items 
Reading : 

Project Director: 
Project Staff: 
Reviewers: 

Writing; 

Project Dlrettors: 
Project Staff: 
Reviewers: 



teth^tics: 

Project Director: 
Project Staff: 
Reviewers; 



Dr. Colleen l^an> FIU ' 

Dr. Barry Sreenbers, FIU; Or. Marisal fiavllon 

Dr. JlwMellSi FIU; Dr. Charier Dzluban, UCF; 

Dr. Tiw Sullivan, UCK 



Dr.- F. J. King, FSU 
Carol Uynch-Brown 

Dr. John Hills, Dr. Laurence Hafner, FSU 



nr. Sordon Brossell, Dr. Jim Hoetker, FSU 
Barbara tehe 

Dr. Dan Kelly, UF; Dr. Nancy Hcfiee, UCFi 
Dr. Owight Burton, FSU (referee); Pamela Laws. 
Tallahassee Cnaniuilty College; Linda Clarke, 
Lincoln High School; Carol Gray, Leon High School 



Dr. T<» Denmark, FSU . 
Dr. Robert Kalin, Dr. Janice Flake, Dr. Halter Dick 
Dr. Mil lias CMtlne, FAMU; Dr. Andria Troutman, USF 
Ms. Pterie Boyd, Escawbia Co. Teacher; Richard 
DeAugiro, Dade Co. Teacher 
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Professional Edi^tlon 

^Ject Directors: Dr.*€aiy Peterson* Or. Oan Kelljf, m 

fievlMrs; Or. If1!11» PlotroMSkl, Uon Oo. Sdiools; 

T^dM of Leon Schools as follows: Jeff 
Patterson* Tead^r^ Wins ton l^rrel 1 » 
PHftclfiBl* Sealy Eleo.; Bet^ Matson* tH^ { 
Sciiool Hath; George Orteoa, ScHool } 
Fraom Stalhtortiit Middle School i Carol 
Sen»ac» £lea.; Mncsy Falnlai, H10I Sdiool; 
Peter LlnkMiy, High School 

Teaching ami Learning; 

Project Director: Dr. ifeUer Mai^r. FSU 
Project Assistant: Jtm H» 

Revleners: Or. It^ Wtchell, DSS. ^U; Ms. Linda flirts. DRS, 
FSUs Or^ <]ohn Hans^. FSU; Dr. loiter D1ck» PSUs 



Plagiosis and Assw«8gnt: 

Project Directors: Dr. /Uma Itelson and I^. Nlllard IteUon. FAU 
ftevleifers: Dr. Helen Franke* Pals teach Oo. Schooiss 
Nary C. &neene« torge Nmiljf, UN; Thais 
Vlllanueva* Tocher, Boca Raton E1e», School; 
Frances Carter , Teacher, Addison Mlmer El en. 
, School I 4^. fbirray !. Gell^t FAU; Frank Martinez, 

Teaciffir» Boca Raton Middle ScIkmiI; I^. Hllllan 
v. Meredith 

Classroois ManageBiCTt/A<te1n1strat1ve Skills; 

Project J^^tor: Dr. Ronald Peake* IMF 

ProJecT Staff: Ihr: Dorothy Diaoond, Ih". Itelen Jasi», Dr. Hilda 

Schaer, Mrs. Carlisle Semes 
Rev1a«ers: Dr. Htlla Crane, IMF; Ih*. Onega fonlner, UWF; 

Mrs. Hazel Scales. Assist. Principal; Mrs. Jo 
Tarwter, Eleraentary Teacher; Dr. James Za Irau, 
IMF 

Itefli Re¥lew : 

» 

Director: 'Dr. Garfield Mil son, DOB 
Coordinator: Dr. Annie Hard* USF 

Consultants: Dr. MIlllBm Mdirens. Professor, Educatlimal Measuresient, 

Michigan Stlite Uhlivrslty 
Dr. Hoimrd Stoker, Professor, Educational Research, 

Florida State 1^1 vers Ity 
Dr. To« D. Freljo, (Formerly Professor, Itolverslty of 
South Flor1<to} President PK Associates, Tanpa * 
Staff Assistants: Christian Doolln, Clara Caste, Hllllam Green, 

Joan Fox 
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tb. Mary Ttw^w. Te9Ct«r. Vtnice Mi^ Sclwol 

Ms. Helen fiooiteon. l^^r, Sawlnoie Co. 

Ifr Fme. Teacto. mrtJtmst m#i Sc»»ol. FInelUs Co. 

Jfrs. Ldtosa SBltht Assistant Superliit^ident ft>r Personnel. Vblusla 

Co. SdK)OlS 

Dr. Bruce Hall, Oialnian, Iteasurwnt « Itesearch Departnent, College 

of Education^ Uh1v««1ty of South Florida . „ 

Dr. Pan Mm Frpftesson CurHoilira. teiiversity of ^uth Florida 

Dr. Kenneth loewe. Student AssesasMit, N»'^VLf?*^!l*?w.«.»*4«« 
Dr. Sabrtel faldes. Omultant. Forel^ Language & Billn^l Mucatlon, 

Omrtaent of Educatton . ^ ^ 

Ms. UlsPlaag, Teacher, OiariferUIn Hlgii School, Hlllsboro^ Co. 
Dr. JKHes Beck* College of EAicatlon, FloHda A * N University 
Dr. Paul Salla^ier, Assistant V1ce4»reslitent, Faculty Relations. 

Flort(te Intematlaial diversity , ^ r n 
Dr. Wldred Huir^. Ccwrdinator of Fe<to«l Pn^nais Evaluation, 

Hlllshorou^ County Sdiools 
Hrs. Itergarer^^^ Teacher, 0e^1e^Br€i*ster El^^ School, 

Vblusia tounty ^ ^ ^ 
Mr. mnm Sanlt. Teachr. Oade Co. Sdwols 

Dr. Annette Soln. /teting Chairperson, Division of Education, Bethune- 

Coolosan (Allege ^ «^ 

Dr. Martan Caltii^, Wrector of Teacher Education, Florida Southern 

Col lose 

Ms. Mary Hewton, Teacher, umem H1^ School, Polk County 

Mr. Larry Kantuck, Hath Supervisor, Jrwwrd Co. Schools 

Dr. Nelson Towle, Coordinator, ftedia & Instructional Material, 

Sarasota County Schools ^ . 

Ms. Quintilla lynch. Teacher, Columbia Cto. Schc«>1s .... 
Or. Linda CitJcker, Oiairperson, Reasearch & Found, of Education, 

College of Educatlim, University of Florida r^...«*4n« 

Dr. Carolyn SmxHi, Associate Dean for Instruction, College of Education, 

University of Miawl 

Field Test ; 

Coordinator: Dr. Thomas Fisher, Administrator, Student Assessment. 
I^rtment of Education 

Test Sites * Administrators 

Bethune-Cookman Or. Annie Ward. University of South 

Florida 

FloHda Agricultural and Mechanical Dr. Thaxton SpHngfleld, l&>n County 

University Schools 
Florida State University 

norida Atlantic University Or. Robert Pickens, Palm Beach County 

Florida International University Schools 
norida Meaorial College 
Palm Beach Atlantic Collei^ 
University of M1a»1 
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Test Sites (Cont.) 
University of f^lorlda 



Dr. Mel Luca$, Alachua County Schools 

Or. Russell J. Crouse, Orange County 
Schools 



University of Central Florida 



Jacksonville University 



Joseph R. Wallace, Ouval County 
Schools 



St. Leo College 
Unfversl*^ of South Florida 



Dr. Susan Turner. Hillsborough County 
Schools 



Sarasota Vo-Tech Center 



Dr. Rick 0. Nations, Sarasota County 
Schools 



Standard-Setting Panel 

Coordinators: Dr. Sarfleld tfllsmi. Departsent of Education 

Dr. Dale Lake, (H*gan1zat1onal OevelMnent Consultant 
Dr. Annie Hard, (h)1verstt;y of South Florida 



Preparation and Premutation of Teclwlcal Materials: Dr. Annie Ward 
Conduct of Group DtsCusslons: Or, Dale Lake 




COTE Members: 

Or. Marian Calti^. Director of Teacher Education, Florida Southern 
College 

Lonnle Colenan, Principal, Dade Cimnty 

Dr. Donald 6111, &iper1ntend«9t of Schools, Volusia County 

Mdrjorle Head, Teacher, Broi»rd County 

Dr. Willis Holcoedke, Provost, Bronard County, Cossainlty College 

Geraldlne Kalvln, School Board Mather, Collier Cotmty 

-Dr. JoNi Lizer, Director of Staff Developsent, Hillsborough County 

Teresa Mention, Teacher, Daite Ciwnty 

6111 Moore, Teacher, Seslnole Coimty 

Laurie Mirray, Teacher, Ouval County ' 

Jaises E. Reese, Principal, Esca^la County 

Dr. Barney Stoutaaire, Director of Iwher Education, University 

of West Florida 
Ida Welmer, Teacher, Brevard County 
Fannie Williams, Teacher, Bronfard County 
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Fomer COtE Menberss 

Hilda Cours^y* Program Plmmlog $)m^a11st> Udiltm County 

Jaa^ 0antt Oeaii of ^ Oolloge of E<&tcatfon» Florida State U..1yers1ty 

(hilntllla L^di. Tea^ier. Coltoibla CDiinty 

Inaa Itoraifi, UQf HsBter 

Bob fMctiburg, Vlro President VaTencIa Conounlty Collofie 

Maty Tmnpson. Teacher* Sarasota County 

Allan Xeeth, Schoo* 8oard Nestor, Semi ml e County 



Observers: 



Chris Ooolln, m 

Fred Milton, im 

ifofm Staples, DOE 

Shelley Boone, m 

Mph TorllngtOfi, DOE 

Claudit Thomas, FTP/KEA 

Mrs. JlsnQf Iteese, EscaoiblB Qounty 

Mrs. Barbara Keeth, Sealnole Ccninty 

Mrs. Ralph Turlington, Tallahassee, Fl 
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APPENDIX B 

« 

Evaluation Strategies 
for 

Florida Generic Coa^tencles and Subskills 
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Florida Essential Generic Cwnpetencles 
Evaluation Strategies 



KEY: X - Minor Evaluation Ei^hasls 
XX - Major Evaluation Es^hasis 







^rformance 


Evaluation 








Mr1tt«» 
Exam 


Content Area 
/^)pl Icatlon 


Generfc 
Application 


Deleted 


lyeimynstrate the ability to 
orally cammnicate informa^ 
tion on a given topic In a 
coher&ftt and logical swnner. 


• 




XX 




a. 


Utilizes the principles 
«f simplicity and 
clarity In organization 
of oral presentation. 




• 


XX 






Uses standard English 
In oral osnminlcatlon. 






XX 




c. 


Uses vocabulary suit- 
able to the topic and 
audience. 




X 


XX 




d. 


Speaks with vol 
and pace that promotes 
cor^n^enslon. 






XX 




e. 


Provides verbal and 
non-verbal cues to the 
organizational struc- 
ture of the oral 
message. 


• 


• 


XX 




f. 


Provides relevant ex- 
amples that illustrate 
oral content. 




m 


XX 




2, Demonstrate the ability to 
write in a logical, ©asiii/ 
understood style with 
appropriate grarmar and 
sentence structure. 


XX 




X 




a. 


Differentiates between 
formal and infonnal 
written English and 
demonstrates ability 
to use both forms. 






• 


X 



ERIC 
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KEY: X - Minor Evaluatlcm Es#as1s 
XX - Major Evaluatlw! Ei!|>hasis 



HrformmGd Evaluation 





UHtten 

iff f t^i^ii 

Exan 


A9Pll€at1<m 


Seven c 
Application 


Deleted 


b. Uses lan^a^ at the 
levei appn^riaiw w 
the topic and reader. 


X 


XX 


X 




c. Coa^rehends and applies 
baste nechanics of 

1ta11zati(»i and pwic- 
tuation. 


XX 


> 






d. Comprehends and applies 
strticture. 


XX 








e. Co^rehends and applies 

the organ 1zati<m of 
written material . 


XX 








T. cwi^rensifls ano applies 
standard English usage 
in written cmnunica- 
tions. 




\ 


■ 




J. DemonstTHte the abilittf to 
messagm after listeniny. 






XX 




a. Accurately follows 
imjlti-step oral direc- 

tiCHIS. 






XX 


• 


b. Listens effectively 
for the orgai^lzatlon, 
fnain Idea, subordinate 
Ideas and details of a 
message. , 




X 


XX 




c. Listens effectively in 
orcter to identify 
relevant informatlcm 
and propaganda 
techniques. 




X 


XX 




d. Listens effectively in 
order to draw Infer- 
ences. 




X 


XX 
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KEY: X * Wnor Eya1uat1<m Emphasis 
XX - Major Evaluation Bi^asls 







Performance Evaluation 


I 




Written 
Exam 


Content^ Area 
Application 


Generic 
AiH)l Icatlon 


Deleted 


e. Siflmarlzes the nessage 
after listening. 




X 


XX 




f . Comprehends both stan- 
dard and nm-standard 
English 1 engage used 
by students. 






XX 




4. Demonstrate the ability to 
rmd, eta^relmidg and inter- 
jpret orally anf in writing, 
profeasitmal aaterial. 


XX 




K 


•■ 


a. Identifies and evalu- 
ates relevant profes- 
sional material. 


XX 


X 


X 




b. Understands basic 
statistical terminol- 
ogy (such as: nean. 
median, node). 








X 


c. Denonstrates literal 
reading skills (such 
as recognizing nain 
Ictea, details, 
sequencing, coi^arl- 
son, and contrast). 


XX 




X 




d. Deaowstrates Inter- 
pretive reading 
skills (such as: 
predicting outcone, 
drawing c(»f)clus1ons, 
making generalizations). 


XX 


X 


X 




e. Dentmst rates critical 
reading skills (such 
as: recognition of 
relevant and Irrele- 
vant Information, 
propaganda techniques, 
and fallacies In 
reasoning). 


XX 


X 


X 
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KEY: X - Minor Evaluation Emphasis 
XX - Major Evaluation Ejiyhasls 



Written 
Exam 



Performan ce Evaluation 
Generic 



Contort Area 
Application 



Application 



Deleted 



f. Produces a logical 
summary Interpretation 
of the results of re- 
search In professional 
material. 

5. Dmamstrate the abilittf to 
comprehend and swr* with 
fandameatal mathesmtical 
cojwepts, 

a. Adds, subtracts, 
multiplies, and 
divides i^ole numbers, 
decimals, and frac- 
tions. 

b. Demonstrates the mean- 
ing and use of frac- 
tions and percents. 

c. Represents and inter- 
prets data using 
charts, tables, graphs, 
and maps. 

d. Solves measurement 
problems Involving 
length, area, volume, 
capacity, weight, time, 
and temperature, using 
U. S. customary and 
n«tr1c units. 

e. Applies mathematical 
skills to solve real 
world problems. 

f. Identifies geometric 
forms and relation- 
ships. 



MX 



XX 



XX 



XX 



XX' 



XX 



*Not validated in professional survey but was retained because of statutory 
requirement (229.841 F.S.). 
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KEY: X - Minor Evaluation Eisphasls 
XX - Major Evaluation EnpHasIs 



Written 
Exaffi 



Pjerfoniance 
Content Area 
Application 



Evaluatlim 



Generic 
/^llcatlcw 



Deleted 



6/19. The ability to coi^pretmnd 
patterns of pla/aical, 
social, and ace^mic 
developamit in studeaits 
and to coufiael students 
concerning ttmir needs in 
theao areas. 

a. Demonstrates knowledge 
of basic principles of 
hunan gnwth «id itevel- 
(^xnent. 

b. (»>ta1ns knowledge of 
stuttents tfirou^ class- 
room tests. teiK:her 
observations, and 
student records to 
contribute to under- 
standing of student 
needs. 

c. Explains test data and 
other classroow eval- 
uations to students In 
relation to tf»1r edu^ 

• catlmal needs. 

d. Applies notlvatlohal 
tec^lques to encourage 
students to be acbleve- 
ment-orlented and goal- 
dlrected. 

e. Assists the stu<tent In 
relating achievements 
and Interests to apti- 
tude and ability. 



MX 



XX 



XX 



XX 



XX 
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KEY: X - Minor Evaluation Emphasis 
XX - Major Evaluation E«?)has1s 





Written 
Exam 


Ccmtent Area 
/plication 


fieneric 
Application 


Deleted 


. f. Demonstrates loionfledge 
of alternative school 
and commmlty resources 
for students who have 
special needs. 








X 


g. Assists stuctents in 
developing Individual 
learning activities. 




X 


XX 




h. Develops student aware- 
ness of career oppor- 
tunities using school 
and connunlty 
resources. 




- 




X 


7, Diagnose. the entry level 
knowledge and/or skills of 
students for a given set of 
instructional objectives 
using diagnostic tests, 
teacher obsrerva 1 1 on , and 
student recfords. 


XX 


X 


XX 




a. Selects the specified 
knowledge or skill to 
be i^agnosed, and 
determines the n»st 
appropriate method 
for conducting the 
diagnosis. 


XX 


X 


X 




b. Selects or constructs 
a test to diagnose 
student learning 
needs. 


XX 


X 


X 




c. Uses classroom ob- 
servation techniques 
to diagnose student 
learning needs. 


X 


X 


XX 




d. Uses Information 

from student records 
to dla^se student 
learning needs. 


X 




XX 
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KEY: X - Minor Evaluation E^iiitiasls 
XX - Major Evaluation Esfifiasls 



Mrltt^i^ 
„ Exam ' 



Pierforaance Evaluation 



Content Area 
Application 



Gfineric 
Application 



Deleted 



e. Interprets results ob- 
tained froBi diagnostic 
tests, teacher observa- 
tion, and Infoimtlon 
frora student records. 

Identify long-ran^ goals » 
for a given ^ubjec± area. 

a. Identifies state and 
district long-range 
^1s. 

b. Fonmilates subject 
area goals consistent 
with state and district 
goals and student 
needs. 

Construct and sequ&MX re^ 
iated short- rai^e objectives 
for a given subject area. 

a. Identifies knowledge, 
skills, and attitudes 
to be attained for a 
subject area. • 

b. Constructs or adapts 
short-range objectives 
for Identified know- 
ledge, skills, and 
attitudes. 

c. Sequences short-range 
objectives consistent 
with comnonly accepted 
principles of learning. 

Select, adapt, and/or devel- 
op instructioml Saaterials 
for a given set of instruc- 
tional objectives and 
stwient learnixtg needs. 



XX 



XX 



Addressed In (9a} 



XX 



XX 



XX 



XX 



XX 
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KEY: X - Minor Evaluation Emphasis 
XX - »tejor Evaluation En«>has1s 



Uritten 
Exam 



. yerforwance Evalua tion 
Content Area 



Application 



Generic 
Applicatioif 



Peletcd 



V, Determines desirable 
characteristics of 
materials based 9n 
objectives and student 
learning needs. 

b. Locates and evaluates 
available Instructional 
material . 

c. Selects iBBterials to 
assist students in 
mastering an objective. 

d. Demonstrates technii^s 
for modifying materials 
to assist students in 
mastering an objective. 

» 

e. Determines materials to 
be developed based upon 
existing resources and 
student needs. 

f. Identifies and selects 
resources needed for 
materials development. 

g. Designs and constructs 
materials based tm 
instructional objec- 
tives, student needs, 
and .available re- 
sources. 

h. After use, evaluates 
the effectiveness of 
instructional materials 
in accomplisWng objec- 
tives, and revises 
accordingly. 

ii. Select/develop and sequence 
related learning ac±ivitie8 
Appropriate for a given set 
of instrmstionAl objectives 
and student learnix^ needs. 



XX 



XX 



XX 



XX 



XX 



XX 



XX 



XX 



XX 



XX 
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KEY: X - Minor Evaluatfim Eiq^ls 
XX - Major Evaluation Cnviiasls 



ferforoance Eva" tatlon 





ExffiB 


Mmcenv wit 
Application 


ewierfc 
/^llcatlim 


Deleted 


a- Cosprehends basic pHn- 

v.i|iic;9 111 f^bmi&l Vyvwf fcfl 

and developnent. 






X 




m 

b. Id9nt1fle;s conditions 
that affect learning* 


XX 


X 






e. Ittentlfles alternative 
activities to achieve 
an objective. 


XX 


XX ^ 


X 




d. Selects an ai^roprlate 
leamlna activitv to 
achieve an objective. 


XX 


XX 


X 




e. Combines appn^rlate 

.Into an Instructional 
sequence. 


XX 


XX 


X 




?. Establisb rapport with 
students in the classrooa 
!>» usir^ vesAal m>a/or 
visual acstivatiisnal 
devices. 


X 


jr 


• 




a. Secures the attention 
of students throu#i 
appn^irlate devices. 


X 




XX 




b. Relates Instructional 
objectives and activi- 
ties to Interests, 
capabilities, and 
exoerlences of «tji- 

dents. 


X 


X 


XX 




c. Informs stiM}«its about 
objectives, subsequent 
learning taslcs, and 
perfonsance expecta- 
tions. 


X 


X 


XX 




d. Explains choices and 
limitations of possi- 
ble learning activi- 
ties. 

1 




n ■ 

X 


XX 
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KEf: X - Minor Evaluatlw Ei^sis 
XX - Major Evaluatlm Emphasis 





Written 
ExffiB 


Conteit Area 
Application 


Generic 
Af)Pl1 cation 


Deleted 


e. Mters instructional 
strategies during 
learning activities 
based cm stuitent re- 
sponses and Other 
factors. 


X 


X 


XX 




f. Relates studKits* anij 
teacher's experlaices, 
thoughts, and feelings 
to learning activities. 




X 


XX 




g. Uses reinforcement 
techniques to assist 
In stiKlent motivation. 


X 




XX 




h. Uses media to secure 
Interest and nuilntaln 
attention. 


X 


X 


XX 




1. Uses student products 
and talent to secure 
Interest and maintain 
attention. 




X 


XX 




3. Present direc±ions for 
carrying cmt an iimtruc- 
tional activity. 


X 


X 

> 






a. Selects appropriate 
means for presenting 
directions. 


XX 


X 


XX 




b. Secures attention of 
students for the pur- 

tlons. 




X 


XX 




c. Informs students of 
objectives, assess- 
ments, and performance 
standards'. 




X 


XX 




d. • Informs students of 
the sequence and 
nature of learning 
activities to achieve 
the objectives. 




X 


XX 
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KEY: X - Ninor EvaluatliBi &fl|^1s 
XX - Major Evaluation E^iphasls 



mttm 



Content Area 



Evaluatlcrt 
Application 



Deleted 



e. Iitentlfles laaterlals 
for a learning acti* 
yity and explains 
their use. 

f. Detemfnes If stuteits 
understand direetlons. 

9. Clarifies directions 
by residing to stu- 
dent questions. 

14, Construct or assflmfcle a 
classroom test to masure 
stwSant perfanaaaoe accozd* 
ing to criteria bas9d vQpoa 
objective* 

a. Identifies uses of 
basic types of class- 
room tests and asses- 
ment techniques. 

b. Identifies appropri- 
ate, uses of norm- 
referenced and 
criterion-referenced 
testing. 

c. Given an objective* 
specifies knowledge 
and skills to Jbe 

. assessed. 

d. Selects appropriate 
assesmit tech- 
niques to evaluate - 
mastery of an objec- 
tive. 

e. Detenelnes lioilta- 
tions, constraints, 
and requirements for 
administering tests. 



XX 



XX 



XX 



XX 



XX' 



XX 



XX 



XX 



*Not validated In professional survey but was retained because of statutory 
requiraient (229.575 F.S.). 
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tCEY: X - Minor Evaluatlim Eiq>hasl5 
XX - Major Evaluatloi &^s1$ 



15. 







Performance 


Evaluation 




Written 
Exam 


Content Area 
AiH>11cat1(»i 


Generic 
Application 


Deleted 


f. Constructs and Identi- 
fies test Items aAd 
tasks that evaluate 
iiastery of an objec- 
tive. 


XX 


1 

XX 


X 




g. Identifies criteria 
for standards of 
performance. 


XX 


XX 


X 




h. Ass^les test conpo- 
nents Inclwling test 
Items, dlrectlims, 
and scoring 


• 


X 


XX 


• 


1. Evaluates and/or re- 
vises tests on the 
t^sls of validity. 
Pliability, and 
student responses. 


X 


X 


X 




Establish a set of clasa- 
room routines axd j^rooe- 
dares for utilization 
and care of mat&cials. 


X 


X 


3CX 




a. Involves students In 
developing classnxm 
routines and proce- 
dures for utillzatlcm 
and care of materials. 




X 


XX 




b. Determines the type 
and amount of nate- 
rlals necessary to 
complete classroom 
assignments. 


X 


XX 


X 




c. Organizes an effective 
system for placement 
and distribution of 
materials In the class- 
room. 


X 


X 


XX 





mc 
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KEY: X - Minor Evaluation Ecoptiasls 
XX - Ibjor Evaluation Ei^sis 



ihrltten 



^rffoffflnte Evaluation 
ConMt Areti Merle" 



tollcatlon 



Application 



Oeleted 



d. Organizes md mrm^es 
a center that will 
serve as a fiocus of 
Interest for student 
learning (such as a 
Nilletin board, display 
table, or exhibit). 

e. Iitentlfles i^lcal 
eleiaefits »id arrange- 
isents In the classroom 
that directly affect 
learning. 

f. Involves students In 
develf^lng routines 
and procedures l^r 
physical laovmnt In 
the classroom. 

g. Arranges classroom 
furniture imd equips 
laent to acconKxtate 
selected teaching 
strategies. 

h. Identifies approved 
proceckjres for move- 
iient of students In 
emergencies that can 
be anticipated. 

16. Fonmilate a staafaitf for 
student behavior in the 
classroom, 

a. Identifies approved 
safety procedures and 
Incorporates them Into 
a standard for student 
behavior In the class- 
room. 



XX 



XX 



XX 



XX 



XX 



XX 



KEY: X - Minor Evaluation En|>has1s 
XX - Major Evaluation Eu^JhasIs 



Written 



Perfb raance Evaluation 
Gaieric 



Content Area 
Application 



Application 



Deleted 



b. Identifies and Incor- 
porates socially 
accepted nom^ (such 
as imjtual respect, 
cons1(terat1on of 
otters, courtesy) Into 
a standard for student - 
behavior In the class- 

c. Ictentlfles characteris- 
tics of the stuftent 
population (such as age 
and maturity) that need 
to be considered In 
fbnnulatlhg a standard 
for student behavior tn 
the classrocHR. 

> 

d. Establlstes a realistic 
standard of behavior 
that has potential fpr 
consistent application. 

e. Identifies and incor- 
porates state and local 
policies Into a stan- 
dard for student behav- 

.-^ ior In the classrMm. 

17. Identify causes of classroom 
misbehavior and employ a 
technique (s) for totrecting 
it. 

a. Identifies factors of 
the physical environ- 
ment that affect stu- 
dent behavior. 

b. Identifies social and 
emotional characteris- 
tics of the teacher 
that affect student 
behavior. 



XX 



XX 



XX 



XX 



XX 



XX 



XX 



XX 
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KEY: X - Minor Evaluation EsQHiasis 
XX - Najor Evaluatlim Esf^sls 



■ ^rfomance Evaluation 





VHtten 
Exam 


Cmtoit Area 
Amllcatlon 


6o»enc 
^llca^lpn 


Deleted 


c. Identifies {>Hys1caU 
social . and enotlonal 
characteristics of the 
student that af^t 
student behavior. 




X 


X 




d Identifies out-of'" 
sctool factors that 
affect student behavior. 


XX 




X 




e. Identifies aspects of 
InstructltMfiaf procedures 

and ^arhn1oiiP<c wh1<*h 

QIIIU IrWiHI l^fifca wiflv'* 

affect student behavior. 


XX 


X 


X 




f . Demonstrates effective 
techniques and strate- 
gies for managing stu- 

(fonfi fatfhavfnr 


XX 


X 


X 




g. Uses selected verbal 
and non-verbal tech- 
niques for reinforcing 
and ffiodlfylno studeit 
behavior. 




X 


XX 




h. I(tent1f1es and uses 
school and cowainity 
resources for assistance 
In modifying student 
behavior. 




X 


XX 




1 • UUV0 lll> wllU Uh 1 i 

parental assistance for 
modifying Student 
behavior. 




X 


XX 




8. Jd&ttify and/or develop a 

system for keepingf records of 
class and irviividual student 










progress^ 


XX 


X 


jr 

• 





G7 



-66- 



KEY: X - Minor Evaluation ElB^Jtiasjs 
XX - Major Evaluation Er^sIS 



Written 
Exam 



a. Cimstructs a system for 
recording Individual 
student knowledge and 
skills progress In a 
subject area. 

b. Identifies wethods for 
reporting 1nd1v1(hial 
student progress In 
knowledge and skills In 
a subject area. 

c. Identifies methods for 
recording class progress 
in knowledge and skills 
In a subject area. 

d. Identifies methods for 
reporting class progress 
In knowledge and skills 
In a 'subject area. 

e. Demonstrates kncwledge 
of the laws and policies 
"governing the content 
and use of student 
records. 

19, (S0V Competency h6) 

20. Identify and/or d&sonstrate 
bBimviors which reflect a 
feeling for the dignity and 
worth of other people in- 
cluding those from otfter 
ethnic, cultural, linguistic, 
and econcmic groups, 

a. Creates a learning 
environment In which 
students express them- 
selves openly and 
honestly. 



XX 



XX 



XX 



XX 



PerforwMice Evaluation 
Content Area Generic 
Application 



XX 



Deleted 



XX 



XX 
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KEY: X - Minor £va1uat1«i Eiii^asis 
XX " Hajor Evaluation' Entasis 



Exan 



b. Assists sttHients In 
understM] ling that 
Individual differences 
enable each person to 
make unique omtrlbu- 
tlons to the grbup 
effoi-t. 

c. Deinonstretes awareness 
of cultural dl^'ferences 
In dress, beliefs s ami 
practices. 

d. Establishes an environ- 
ment for positive ccei- 
nninicatlon and Inter- 
action betMeo) stuitents 
from different soclo- 
cultural backgrotmds. 

"^2. nmowtxate instructimial 
and social skills which 
assist students in devBlop' 
itig a positive seXf-conctapt. 

a. Exhibits behavior In 
the classroom that Is 
einpathetic, positive, 
and reinforcing. 

b. Assists students In 
initiating self- 
directed learning. 

c. Assists students In 
understanding their 
needs, motives, expe- 
riences, and individual 
value and dig^ilty. 

d. Selects and uses cur- 
riculum materials In 
accordance with the 
abilities and mastery 
levels of individual 
students. 



fferformaice Evaluation 



Content Area 
Application 



XX 



Generic 
Application 



Deleted 



XX 



XX 



XX 



XX 



XX 



XX 
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KEY: X - Minor Evaluatlw Emphasis 
XX - Major Evaluatlcm Emphasis 



Written 



22. D&mmsteat0 instructicnuil 
and social slciJIs lirhich 
assist atudants in inter- 
acting cmtxuctivelt/ with 
their peejrs. 

a. Establishes an envl- 
romaNit that iiermits 
students to cooperate 
and shai^ 1(teas and 
ioaterlals. 

b. Assists stud«)ts In 
applying cwistfuctlve 
criticism In response 
to each otters' work. . 

c. Establlstes a learning 
envlnwiBient desired to 
assist students In ex- 
hibiting positive Inter- 
personal traits (such as 
mutual respect and coop- 
eration). 



d. 



Uses techniques that 
assist students In exam^ 
ining their values, 
attitudes, and beliefs. 



23. Demonstrate teachix;^ skills 
which assist students in de^ 
veloping their own values , 
attitudes, and beliefs. 

a. Assists Students in 
understanding the need 
to explore alternative 
solutions to problems. 

b. Establishes teaching 
strategies that allow 
students to nake choices 
based on clearly defined 
consequences . 



Perfors Mice Evaluation 
Sendrlc 



Content Area 
Ap pllcatiim 



ApplicatiWL 



Deleted 



XX 



XX 



XX 



XX 



XX 



XX 



XX 
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/y>PENOIX c 
coverage of the Examination 



-71- 

Covarage of the Examination 



during the process of review and developra^t described In Chapter I, some 
subskllls Mere deleted aod a few of the cffii^tencles and subskllls were judged 
not to be assessable by a wltten test.. Nineteen of the conpetencles, covering 
80 subskllls, have been jtMlged to be msurable to sose extent by a written 
test. Appendix B Indicates those subskllls Mch were deleted ami, for those ' 
which reraain. It Indicates the prinary assessaent technique to be w»i*hI. 

The Teacher Certification Cxoif nation covers those co^tencles and subskllls 
checked In the first colunn of Appendix B. The examination consists of four 
subtests: three In Geieral Education (reading, writing, and mathmtlcs) and 
one In Professional Education. The coi!9>etencies to foe covered by each subtest 
are as follows: 

Reading • Ctxi^tency 4 

Writing CfflJ^tency 2 

I^th^Mtlcs Copyfttftfify S 

Professional. Education Coi^tencles 6^7, 9. 10, 11, 12, 13, U, 

16, 17, 18, 20, 21, 22, and 23. 

Examination Schedule - 

t 

The exam1nation_^ill be scheduled to cover a full day. A typical exaaination 
schedule in1gRt[be as follows; 

8:00 - /8:30 Check In, including' retakes for Writing Subtest 

8:30 - 9:00 Oeneral Instructions 

9:00 - 9:45 Writing Subtest (Production Task) 

9:45 - 10:00 Break (Check M retakes for Hathematics and Reading Subtests) 

10:00 - 11:00 Nathematic Stmtest 

11:00 - 12:00 Reading Subtest 

12:00 - 1:30 Lunch 

1:15 - 1:30 Check retakes for Professional Education 

1:30 - 4:00 Professional Education 



Ex^i nation Dates 

November 22, 1980 
April 4. 1981 ' 
auly n, 1981 

Exaai nation Locations 

Eight sites have been identified for the 1980-81 year. 

University of North Florida, Jacksonville 
University of Florida, fiainesville 
Florida State University, Tallahassee 
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Unlverslty of Uest Florida. PensacoU 
Un1yers1l;y of South Florida, Taiapa - 
Un1ve**s1ty of C^tral Horldat OrUndo 
Niani Dade Consin1t;y Collm, ^th, Miami 
Edison €oinnin1ty Collesfo. Ft. Hyers - 
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The Subtests 



The Florida Teacher Certification Exanlnatlon is divided Into fcNtit subtests: 



Reading. Writing, Nathenatlcs, and Professional Education. 

1. Reading: 

a. The Reading subtest uses the nultiple-cholce ClOZE procedure. 

b. The test consists of eight passages of awroxlmately 100 wwds, 
selected from the foil caring sources: (1) textbooks comonly used In 
required professional education cmtrses. (2) Journals and newsletters 
publIsM by non-specialized teacher organizations. (3} docuoents for 
t«achers-1n-^neral produced by the State D^rtnent of Education, and 
(4) teachers* manuals for tests In coDson usage In Florida. 

c. Each passage has ten words deleted. Deleted words are nmins. verbs, 
adjectives or adverbs* not structural words. 

d. Examinees are asked to select the word to fill each deletion from 
among four choices lAlch are syntactically equivalent but different 
in ffleanlng. 

2. Writing : 

a. Each examinee wrftes on one topic, to be selected from two options. 

b. All topics are required to meet these criteria: 

1. Self-explanatory (I.e., clearly and explicitly phrased) 

2. Defined and limited 

3. Familiar to every examinee 

4. Stimulating 

5 . Fresh^ 

6. Of middle emotional ground (I.e., neither too peitestrlan nor too 
sensational ) 

7. Non-biased and non-biasing 

c. Scoring Is holistic (general lBq)ress1on1st1c). 

d. Judges have been specially selected and trained. They are required 
to reach a specified criterion of agreement with other jud^s {80X). 

e. Ratings are on a 4-po1nt scale, with the score being the sin of 
three ratings. 
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MathCTiatlcs ; 

a. The Mathematics subtest consists of approximately 40 multiple-choice itans, 
divided approximately as follows: 

SOX 

m Real World/ 
Co^utation/ Prohlem 
Understanding Solving No. % 



Subskills 



(1) Adds. Subtracts, buUI- 
plies and divi<te5 whole num- 
bers, decimals, and fractions. 

(2) DeiBonstrates.the laeanlng and 
use of fractions and per cents. 

(3) Represents and Interprets data 
using charts, tables, graphs, 
and maps 

(4) Solves measurement problems in- 
volving length, area, voltaic, 
capacity, weight, ti^ and tem- 
perature, using U.S. customary 
and metric units. 

(5) Applies math^tlcal skills to 
solve real wrld problems. 



15 



3 
2 



19 481 

6 m 

3 B% 

6 14f 



20 



20 



40 m% 



b. Additional Recommendations: 

fREAi miti 

50% items are teacher related and 50% items consumer related/problem solving 

33% items can be ansi«red by naking a "ball -park" estimate 

16% items contain extraneous Information , ^ a 

30% items have information presented In a graph, Uble or drawing 

50% Items Involve two or more steps 

16% items are analysis or synthesis tasks 

IMEASUREHEHT ITEMSi 

50% English and 50% metric with at least 12 Hems distributed across the 
following topics: length, weight, capacity, area/per 1n«ter/ vol un«. 
elapsed time 



36% Items have "none of the above" as an option as indicated In the item 
specifications _ , ^ 

12% Items have "none of the above" as the correct answer (6 Real World, 
6 Computation/fact/ understanding) 
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Professional Education ; 

Exanlnees have 2 and 1/2 tmn to c(xq>1ete the Professi^l 
Education portion of the exan. 

The Professional Education subtest consists of approximately 100 

ssfltlple-cholce Iteas. lt» dlstrlNjtlon of item iflll be based 
on th^e dmslderatlons: 

1. All c^ei^»c1es'6-23 are eovered. Conpet^ica^ 8 Is addressed 
by the ex^l nation Iteos related to Competency Subsklll 9a. 
Cflnpetenclei 6 and 19 Mere combined by statute. 

2. The extent of the coverage Is based on the priorities set 
by the Professional Task Force. 

3. The coveragens based on a balance be^«een the aajor 
categories ofi the content base and the ccbipetencles as Indi- 
cated below: 3 
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i 








• 
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Dll Expert Rflvlair Panel 

a 2 Staiulard-Settlng Panel 
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OHffnutfoii to 1MK Is K^r^i Tf»aibicfc, ^^i^tttwi i»Mii9 



Bin ^ ll:O0 Tiil I 

llMfyf«M»l« Blat Rnrlpft 

Tiilt fopctf / Hir^rani - Mwrl* Hill 



11:00 p.m. ' 1:00 p.ii* imM 

1:00 - S:00 f TitI If 

(Nftttrlalt •mmoM In o^miw* Tim 1i Mterlili 
for e«€)i milon i» mm «§ mt^liU art ceiipltM* } 
Suff vlll co11«U» Mit«rl«1f for Mdi Itaw mvlew ii^ 
tdtntffy ium to fwHtlOA. 



Mom Officf BoHdlf^ 
Is 30 * t2:00 p^m. £tm%1nm U%k II Oi ilitVMHS 
11:00 - 1:00 p.m. imM 

IrOO p«9. * 5:00 p^m, ^iinm Task II, Oogfn 7Hk III 



• 



Suff will btgfn rwlav of f tan at noim» Tiaaday 
(or as toofi as roHavs ara m^latod flar a t^t 
araa). 
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ERJCi a A - SsOO p.«* Itol III / tmeriUp m^fiim 



STiUT ; OarflalU Iff Or. Annft Hard, C^it Ooolfn, ^ 

pmSOlTASTS ; Or. VilHa^fMr^Sp ^r^ftssor 
EdiicaltMil HAsurvavnl 
m Erld9ft«s Kill 
filefci^ Stat* itofvtrtfty 
Cist LiMlfifl* W^loaii Wti 

1^, KoHird Sls^, PfV^ior 
. Horldi Stalf ttnfvtrflijr (on laavt) 
Ctarymt AaaffMwi: 

ITS Mmu 

mi Aaamraa •aad, ii.C. 
I Atl^siot OiOfffi 303M 

Or, To« rraljo. f^fdtnt 
n^mt^i SOvtlcwt. I Svaluatloti 

AsaocfataSf Im* 
P.0* Soa 17m 
1^, Horfdi I3SB2 



fimaan* Clara C«(^» JiNii H$ 



smm^ 

Us. Minr THoqpson 
\NMfca Hidi Sctool 



lls# ttelam Goods on 

ton tut 
L^^MPod. FloHda 



smo 



Oflft im Una, «^ 

St. l^ttrOi^ Fli*» SI702 

nrs. UHbaa Sa1ti> 
SuaaHot^daot 

Panomial 

Volitsia dmt^ Schools 
Mand, FloHdi 32720 

Or. Omca Hall, Oialraaji 
ftoistraaaffit i f^rsfircli tmf. 
Collai^ of Id^lfon 
Univ. of Sootfi Florid 
TsRipa. norftfi 32&?0 

1^, Din Pwmkmt Profttwr 
C^Mooa of C^icatfon 

of SooU^ riofftfa 
Taqpi, rioHdi 3M20 

t^* X«m»tli LoaMi 
Student Assossamt 
Oapartamt of £doc#ti09» 
Tallahassot. Fla. 32301 

i^, Qabrlol Vafte 
CoDSitltaot 

^rafgo Ijmmm Ml BIHi^nmI 

IdMcatfoA 
O^rtoMt of Idnatfoa 
Koott tofldlDg 
Tall^saot h ymi 



Hi* lota naM 
Jmm^ 3»03 

ftr. Man ^1 

CO II 090 of £ifciGatloo 

tiiUaliafito. fla. SI07 

Ar* l>i«l 6a1la^ 
Asst. ¥lco Prmii^t 
Htu^^ Helallona 
ria. lotamoatl^l ttfilt. 
maul, Florida »m 

01*. f^ay 
Fad. frograos Cvaloatloo 
Nflf^mno County Scteelft 
m E. Orfve 
Taa^» Florid! 33m 

nr%. Har^rat Vol fa 
VIS foacfi^ 

Oe«piio Orawitar llanvntary 
fiiU»^. Florida 32720 



nr. ivovwfi Saidt 
921 Falls Trw Lana 
Pmtfvok Pffias, Fla* 



33024 



Or, Aontttt ColOt Olr* 
Tta^ior location 
Oattm>o^Co(Naisii tella^ 
jS40 Second Avanvt 
fiaytsiia Bosch, Fla. »OtS 



BEST PHTV f 



Or. Mirfaii Cslisiy 
Florida Southifii Callow 
Ukalan^, Florida 33^3 

9^* Hsry NMton 

HI oh SchSOl 
KatMoos, Florlds 31849 

llr« iarry Wantucl 
fbtJi. Si^rvlsor 
fifward (om^ SdMls 
F,0. Ooa H08 
Ft* Uodordalo* Fla. 33319 

Or. Nalsosi 7o«lo 
Coordinator 

Madia I Instructional Hat. 
Sarasota twmt^ ScHooli 
2418 Hitton Straot 
SarasoU. Florida 33S77 



Ki. Qiilntllla ijnch 
F.O. toa m 
%rm^for0, Florida 



Or llnds Cmltr 
Clialrparson 

KostfiTli I Foond< of Cd. 
Collo^ of CdotatiM 
ItolwYlty of Florida 
$alnH»l|l0t Fla. 32S}i 

Or. Canslyn Gsivood 
^soc* Ooan of Instriactlor 
Collooi of Education 
itolvmity of mMi 

Mlaal. Florida 33124 
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i 

Br. Xm 9m»^rk 

219 C^l^Oft BiifldlM 
n9rl4A suit teilwnily 

TiliAiiattM. fu. nm 

m 

man 

CiifTlC4»li« i Imttfuct^ 
Tili«iiaftft««» fU. saw 



Br, f . J* Mi^i 

nof14i SU^ UsIirtrsfUr 
TilUBttm, flu* 32^ 

Br* Stftrd 
307 %tmm mWn^ 



2 



ftp, Ury ¥»ttnm 

center fiw E&catlMtl ftOtnplogjr 

1.A Tuny 

flerftfi SMtt miwrtUy 



Dr. Wflldrd »t)Mn 

College of Eduutfofl 
n«r«d« AtUncfc Univ. 
Boca tatpn. Fli. 33431 



Or. Anna Nelson 
Foundation* ^rtwit 
CDtlege of Muc'tlon 
noritf* AtUntlc ttB«v. 
Boca town, n«. 33«3l 



Profwslwial Etfacatton « CoBiPBtOTc <« 15. 18. 17. 18 



Or. tonaltf P««»« 
El em. S Sec. EA»c«llon 
lfc>«¥, of tMft Florida 
Ptosacola, fJa. 32504 



grofeyt onal Cdt«-*V 5»» - <;«roetenc<es 9. 10. Ui il 
Dr. Walter Haijer 

W Stone Sutldln? I 
nor»dtf State University 
TaMahaisef, Fla, 32306 



V 



SO 

o ' EISTCOrVAViyUffiLE 

hKJC 



T 

FLORIDA 
C E RT I F 



TEACHER 
I C A T I 0 N 
NATION 



• ITEM 
REVIEW 
TASK 
FORCE 



January 27-30, 1980 




thp^rtm^t of idtJffVM 



TASK I GROUPS 



U •'itheniatlcs (Yellow) 



iidntuck 
Sank' 

W. Nelson 



Fraze 

Goodson 

Murray 



Assessment < Green) 



Hall 

Gallagher 
King 



Lo^e 

Crocker 

Peterson 



Teaching, Learning, ^tethods, 
Classroom Procedures (Blue) 



Purdom 

Calway 
Thompson 
Towle 
Newton 



Denmark 
Coins 
Lynch 
A. Nelson 
Valdes 



0 



Personal, Social, 
Academic Development 



(Pink) 



Garwood 
Fox 
Sniith 
Peake 



Beck 
Wager 
Plaag 
Wolfe 



ITEMS 
IN 
PRIMARY 
GROUP 



102 



115 



150 



124 



mmrM 

PRIMARY 
ITEMS 
PER 

PERSON 



51 



58 



41 



41 



M 



Nl^ER 
OTHER 
ITEMS 
(I-M-2 or 
-P"2) 



26 



11 



12 



12 



W 



17 



13 



T 

IT 



22 



21 



C 

2l 



16 



16 



17 



2Z 



READING 
PASSAGE 



TOTAL iirm 

PER PERSON 



1 



51 



58 



41 



41 



47 



46 



47 



00 . 



i/wK II Groups 



'nui ray 
Gallagher 



ite) - 

II 

Kill 
Crocker 



STOK£f> 



XTEI^ TO BE 
REVIEWED 

3 Reviews 
fer Item 



Alt 



ITEM R 



ITEM FORHS 



Reading Booklets/4 Forms 
Math Booklets/ 3 Fonns 
PSA 6ooklets/3 Forms 
T & L Booklets/2 Forms 
AS Booklets/3 Forms 
D A Booklets/2 Forms 
Writing Study Report 



REVIEW FOm 



REVIEW FORMS 



II-M-R-1 
n-M-M-1 

n-«-p-i 



II-M-R-2 
n-M-M-2 
n-M-P-2 



N 



H-M-M-l 



II-M-W-2 



Bias 

Valdes 

Smith 

Beck 



(Gold) - FREIJO 

Newton 
tolfe 



AL 



a 

CM 



Test Booklets: 
Reading 4 
Math 3 
PSA 3 
T & L 2 
AS 3 
DA 2 
Writing . 
Instructions 



II-B-R 
II-B-M 
n-B-P 

It 
H 

9 

II-B-W 



/ 



Content 



MEHRENS 



ERIC 



mAtR ^ 

(fellow) 


Group C 
t>SAD 
(Pink) 


Group B 
TLM 
(Blue) 


Group A 
(Irien) 


Wantuck 


GarMOOd 


Purdoni 




Fraze 


Plaag 


Towl e 


King 


Goodson 


Peake 


t^jlns 


Peterson . 


W. Nelson 


Wager 


Denmark 




Sank 


Calway 


A. Nelson 








Thompson 








Lynch 




bi 









Four Gpups 
of Itemis: 



AS ASSIGNED 
TO GROUP 



Math 
Group A 
Group B 
Group^ C 



102 
115 
150 
124 



II-C 

II-C-M 

II-C-P 

II-C-P 

II-C-P 
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AGENDA 

COUNCIL ON TEACHER EDUCATION 



June 24-26, 1980 
Wakulla Springs Inn 



TIME 



ITEM 



WHO 



Twesday, June 24, 1980 

7:00 P.M. 
7:15 P.M. 



Registration 
General Session 

Call to Order 

Introducticms 

Introduction to the Process 
Presentation of Tentative Recosr- 
stendatlon 

Identification of Faraiaeters 



Chris Doolin 

Marian Calvay 
Garfield Wilstm 
Dale Lake 
Annie Ward 



Dale Lake 



ACTION REQUIRED OR TAKEN 



Security Statements 



Wednesday, June 25, 1980 
6:30 A.M. 

9:00 A.M. 

10:00 A.M. 
10:30 A.M. 

11:15 A.M. 

12:00 Noon 
1:00 P.M. 



General Session 
Mixed Role Groups 
Break 

General Session 
Role - Alike Croups 
Lunch 

General Session 



ERIC 



8b 



Annie Ward 
Group Leaders 



s 
f 



Dale Lake 
Croup Leaders 



Annie Ward 



Receive technical materials for 
Recoi^ndatlon §1 

Modify Recocntendatlon #1 and 
Rationale 



Develop Second Draft of Reconsaenda-* 
tion 91 

Check Recommendation ^1 Against 
Decision Matrix 



AllIuu mi ntmi i Mu ew dettm tf i ' 



Receive technical materials for 
Recommendation ^2 
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TIME 



Wednesday, June 25, 1980 
(continued) 

2:00 P.M. 



3:00 P.M. 
3:30 P.M. 

4:15 P.M. 

5:00 P.M- 
5:30 P.M. 
7:00 P.M. 

7:30 P.M. 



Thursday, June 26, 1980 
8; 00 A.M. 

9:00 A.M. 



. 9:45 A.M. 

ERIC 



AGENDA 

COUNCIL ON TOCHER EDUCATION 

June 24-26, 1980 
Wakulla Springs Inn 



ITEM 



Mixed Role Groups 
Break 

General Session 
Role - Alike Grot^s 

General Session 
Dinner Break 
General Session 

Mixed Roie Groups 



General Session 
Role - Alike Groups 
Break 



WHO 



Croup Leaders 



Pale Li^e 
Group Leaders 

Annie Ward 
Group Leaders 



Dale Lake 
Group Leaders 



ACTION REQUIRED OR TAKEN 



Hodify^econnwndation 92 and 
Rationale 



Develop Second Draft of Rafcw- 'cnda- 
tion #2 

Check Recooanendatim #2 Against 
Decision Matrix 



Receive tecluiical nwterials for 
Recooim^dation 93 

Modify Recomn^ndation 03 and 
Rationale 



Develop Second Draft of Recomnenda' 
tion n 

Check Recoimnendatlon ^3 Against 
Decision ^Jatrix 



8ij 



acenha 

council on xeachek education 



June 24-26, 1980 
Wakulla Springs Inn 



TIME 


ITEM 


WHO 


ACTION REQUIRED OR TAKEN 


Thursday » June 26, 1980 
(continued) 








10:15 A.M. 


General Session 






iuinie wara 


Receive technical s^terials £or 
Recosffi^datlon H 




Luncn 






m 

12:30 P.M. 




Group Leaders 


Modify Recom^ndaClon H and 
Rationale 


Is 30 P.M, 

■ 


General Session 


Dale Lake 


Develop Second Draft of Recorasenda- ^ 
tion H ' * 


2:00 P.M. 


Rol€ - Alike Groups 


Croup Leaders 


Check Recosff^ndatlon #A Against 
Decision Matrix 




Genexel Session 


piarian uaivay 




3:30 P.M. 


Adjourn 




4«fV04i ^0% ail tec^mn* 








0 
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APPENDIX £ ' 
Fonas 

Contractors', Review Forms 

Expert Panel Review Forms: 
Content, Technical* Bias 



Reviewer's Perspective 
(Check all which apply) 

Classro{»n Teacher 



— Content area scholar 

and/or professor 

Measurement Expert 

Ethnic Group 

— Black- — Hispanic 

— NonHispanIc White — Other 
Sex: Male Female 



APPENDIX E.l 
ITEM REVIEW 
Report Form 



Con^etency 

Subskill 

Item No. 
(If more than 1 
for subskill 



(Circle One) Comment if m or 7 



1. Relevance to conq)etency and subskill 

a. Does the it&n address the appropriate 
knowledge base for the cojiqjetency and 
subskill? 

b. Is the knowledge base addressed 
correct and current? 

2* level o f behavior 

a. Which level of behavior (foes this 
item require? 

Knowledge Application 

Problem Solving 

b. Is the level of behavior appropriate 
for the competency and subskill? 

3, Adequacy of item stem 

a. Does the stem make clear what is 
the task to be performed? 

b. Is the wording of the stem clear? 

c. Is the stem free of extraneous 
or confusing materials? 

• Relevan ce of s cenario (if provided) 

a. Is scenario realistic? 

b- Is scenario ap-propriate for the 
coifipvtency and .subski 1 1? 



YES m ? 



YES m 1 



YES NO ? 



YES NO 



YES NO 

YES NO 
YES NO 



? 
? 



YES NO ? 
YES NO ? 



-89- 



93 




-90- 



{Circle Pro) CoaiTOnt If NO or ? 



5. Relevance of pictorial roateHal 
(If provided) 

a. ' Is pictorial material appropriate YES NO ? _ 
A for the C08!petency and subskill? 

b. Is pictorial material necessary VES NO 7 
for the item? 

6. Appropriateness of item options. « 

a. What is the correct ansnwr 

b. Is the correct answer, stated YES NO ? . 
accurately? 

c. Are any other options also correct? YES NO ? ' 

d. Are all the options grammatically YES IK) ? ' 
•parallel? 

e. Are all the options logically YES NO ? . 
parallel? 

f. Are all options plausible for YES f«) ? 
naive examinees? 

g. Are all options free of YES NO ? 
irrelevant cues? 

h. Are all options of approximately YES NO ? 
the same length? 

1. Are the options reasonably YES fW ? 

ordered? - 

7. General it em characteristics 

a. Does the item fit the item YES NO ? 
specifications? 

b. Is the Hen consistent with re- YES ? 
quired formatting? 

c. Does the item have "face YES NO ? 
validity?" 

d. Is the reading level appropriate? YES W ? 

8. Item Bias 

* a. Does the item contain any informa- YES NO ? 

tion that could be seen as 
offensive to the culture, race or 
religion of teacher applicants? 



ERIC 
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Idrcl^ tee) CfliPent If WO or ? 



b. Does the Item contain any Infonna- 
tion that could be seen as 
offensive to either sexual group? 

c. Does the Item Include stereotypic 
deplctlwis of any cultural, jradal* 

. or religious group that are debasing? 

Does the Item Include stereotypic 
depictions of either sexual group 
that are debasing? 

Does the Item portr^ cultural, 
racial, or religious groups as tm- 
equal In ability or natural 
endoMment? 

f. ' Does the Item portray sexual groins 

as unequal In ability or natural 
endowments? 

g. Does the iVem contain clues or 
Information that could be seen to 
work to the, benefit or detriment 
of any cultural, racial., or 

Inus group? 



h. 



rjpllgii 



YES ND 7 



YES NO ? 



YES NO 7 



NO 7 



YES NO 7 



YES NO ? 



Does the Item contain dues or 
Information that could be seen tc 
work to yie benefit or detriment 
of either sexual group? 

Does tim Item contain group-specific 
language or vocabulary (e.g.. culture- 
related expressions or slang}? 



YES NO 7 



YES NO 7 
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APPENDIX E.2 
Panel Revteif Form 



Passage Code 



Passage and Items 



Key: 



1 
2 
3 
4 
5 
6 
7 
8 
9 

10 



Appropri 
Inport 



ateness (Hdm 

Ji^rtant Is it tJttt 
prospective teachers 
be able to read and 
understand material 
such as this? 



1. 
2. 
3. 
4. 

i 



Essential _ 
Inqsortant 
Acceptable _ 
Questionable 



Difficulty level 



1. Easy 

2. Medium 

3. Hard 
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9G 



form I-M-1 
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TASKI 



M&THgMATICS 



ItEM 



Ztem Code 



Key 




Traceback 

Conipetency 

Subsklll 

Item 

Descriptor 



Appropriateness {Him 
Important Is It that 
prospective teachem 
be able to answer 
this Item correctly? 



1. Essential 

2. In^rtant, 

3. Acceptable 



4. Questionable 



Difficulty level 



1. Easy 
Z. ^dlum 
3. Hand 
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erJc 



Item' Code 
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NATHEMATICS 



ITEM: 



ropHateness (How 
nportant Is it that 
prospective teachers 
be able to ansvi^r 
this item correctly? 

1. Essential 

2. Iiqiortant 

3. Acceptable 

4. Questionable 



Difficulty level 



1. Easy 
2t MediiAQ 
3. Hard 



FRir 



9S 
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PI^OFCSSIfflJlL EDUCATION 
ITEH 



V 



0 



Item Code 



Correct Answer 



^on^tency * 

Subskin . 

Content Base 
Category(1es}: 



XI 

III 

lY 

V 

VI 



Apprc^rlateness (HCM 
Is^rtant Is It that 
teachers and teacher 
cand1<tete$ bd able 
to an5*<wr this Item 
correctly?) 



1. Essential 

2. Important 

3. Acceptable 



4. Questionable. 



Difficulty Level: 



1. Easy 

2. Medium 

3. Hard 
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Form I-P-2 



Item Code 



• PROFESSIONAL EKICATIQN 



ITEM 
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Correct Answr 



Appropriateness 
1s9K>rtant It Is that 
prospective teachers 
be able to answer 
this Item correctly?- 



1. Essential 

2. Iinportaht 

3. Acceptable 



4. Questionable 



Difficulty Uvel: 



1. Easy 

2. 'rtidluin 

3. Hard 



» 



Millet 
Accept • 
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Iff* 



4« is iHi mdffif Itytl tpprv- 

b. Is %h9 «QNffi9 of Urn utm 
citirf 

c. Ift the slM frfet of con** 
fusing MtirliUT 

6, 0001 tht %tm wnuin 
• tutrintoift ftnUrtoUT (Hm 
^ rtfvlrtd Iqr tUt tfMciflMi^on* 

l» IHt pIctoHol fultrlil 
iopn^lito for tut fubiMIt? ' 

b. Is t9» oUtoHftI Mltrlil 

AppnyprlittUss pf <t«s oo^li^ 

o. Is XM stylt of tte options 
corrtci ond cltorf 

b. Arc tht optlow trr^ftpotf to o 
Sftd f/«tt9tttfc ordtrf 

s. imst li tht lovtl of 
behavior rtti*<r»0 |y tbt 
Ut«? (Chta M^btst*) 

b» Csn tht 1tt« bt s ws mrt d 
»fth I *b«l1 psri* ftStfuotoF 

c* Moif MiQf stops trs rtqiHrttf 
to solvo tht prvbloiiT 

d. Is tht Itfto *llto1 l^rir? 

t. If ""fteil tN>r1tf,' Is n 
Ttichtr-8tUto4T 
C^wntr^reUtotfl 



YIS W f 

n% m r 

m m t 

m m f 

m m r 

res w t 

m «D f 

res « r 

m m I 

rts « f 

res RD f 



COfl^lAtUMI ^ 



rfS 



s f } 



|1| Ttichtr-8tUto4T 

6. If this S wtgrffSiBM 4 
Vts, »n»i«r '^s' and *b* 



UtoT If 
htlow: 



a. !i It (I) Cnflfsh Sfstmr 

U) «rtr<c SrsttaT 

b, VMch fopic(s} Is (art) 
*(10nnst4T 

(S) Ufi^th 

{2} Mftfi^t 

(3) Capacity 

(< ) Afta/^fff Inpttr/yoliM 

(5) llapitt Tl«« 

7, Is "Hotm of tht abovt* wstitf at 
an option? If tfts, ansnitr "p* 
tfiO ''b" btlow: 

a. Is "iftnt of tht ahott" tht 
corrtct tf»s«trl 



m 
n$ 

YCS 

res 
r£s 



m 

m 
m 



o 

ERIC 



finite. «Mct MtM? 



YIS NO 

m m 



r 



Soi^ltitir Co4t 



#r^ltii SofHop 



101 EEH K?V ' 
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mm 



prtift luMMlt^ for the 

1^ Is titt kn*»*1t«|t tan #iMrt»»td 
corrtct HI* offTwtt 

i. imich 1ml of Wiitrlof itfoti 
this Ittfi nqyirel 

It tH« Itvtl of bohiwlor 

S. ftiiffflui^ of <to» %im 

I. dots tilt »tt« wko cJ«ir Ktat 
If tHo u%% ^ l>9 ^rf^KTHid? 

1^ Is ^ wontlnv tits tt««i 
elssr? 

Is thf it«i ff^ ^ 
At«HiS wr confminq ■stsHalsf 



4, 



fts1g»fi« of ^cnir4o eif omtl^) 

s. Is scmrlo rstlfstUI 

Is sctniHo s^^r^litt for 
tht cospstOfiQr ond »i**kflll 

lif orovioto} j 

i. 2s pUtorls) nstoHol sppro- 
prists for thi m^ttftcy siid 



1% pictorlsl nstorlsl watsssi7 
for tilt 1ts«t 

«. jlpcrrarlsttn gsf of Its* opt^o^ 

s. Is tHs k»y94 Bmmr stststf 
sccursttlx? 

Art snr ot^tr optlei^ siso 
corrtct? 

€• Art s11 tht ootfofis qrm^ 
ftttlcsU/ p«rs)lslT 

4. Art ill the ootloni Ip^^cslly 
pirslltl? 

t. Art all options plsuslblt for 
nsivff easwioetsr 

f/ Art sll motions frtt of 
<rretev4nt diits? ' 

Art #11 optlonf of soprosl- 
«#tely tN sffst Itngtht 

Ji. Art tt»t options rtssonsbljr 
orotrsO? 

7. tenersl it^ c?Mractff<st1c» 

i. Decs tl» Uefli fU tN Ut* 
sotclfititlonsT 

b* »9ts iwr ItesJ bs»t "fsct 

c. I5 tHf rviifint Itvt' spono- 

prfatt? 



• ii 



ItfBTMtr _ 



ftiljfs^ AiBswsir 



TO m I 



1^ W T 



Hnowltapt 
AppllcitfM ^ 
PnbiM Solving 

fES » t 



ttwwmt if or *r 



ftovlCMrs Cote 



■r 



m Ml 

ns m 

m m 

TEs m 
ns 



m ID T 



m fa f 

TW NO T 

VK NO ? 

m m J 

TES NO / 

TES NO ? 

ns » T 

TES NO T 

lES M> ? 

TES « 7 

TCS NO 1 



102 BEST C9?Y pL"51f 



Accept 



HltM>r ^visions _ 
Major Revisions \ 
Reject ' ■ 



f 
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Topic Code: 



Revlaiers Code 



TASK II 
HEASURDSHt REVIEW 
WRItirffi - I 



INSTRUCTIONS; 



Circle tee 



Cowroents 



a. Is the tone friendly and supportive? 

b. Are the Instructions clear about the 
basis for the scoring? 



YES HO ? 
YES NO ? 



c. Are examinees encouraged to plan and YES W) ? 
organize their thoughts before wrltlt^? 



TOPICS: 



a. Is the t<ip1c self-explanatory; that 
is. Is It clearly and explicitly 
phrased? 

b. Is the topic adequately defined and 
delimited? 

c» Is the topic likely to be familiar 
to every examinee? 

d. Is the topic stimulating; i.e., will 
it evoke examinee interest? 

e. Is the topic fresh ; I.e., Is it 
one which has not been over-used? 

f. Is the emotional tone appropriate, 
i.e. , neither too pedestrian nor 
too sensational? 



YES NO 7 

YES m ? 

YES m ? 

YES NO 7 

YES m ? 

YES W) ? 
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r 



Topic Code 
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TASK XI 

MEA$URE>eiT REVIEW 
WRITING - 2 



Reviewers Code 



In answering these questions, refer to the paj^r "Validation of Topics and Comparison 
,of Three Presentation Modes for the Writing Subtest of the Florida Teacher Certifica- 
tion Examination." 



I. Topics 

a. Do any topics yield lower 
ratings than others? 

b. Do any topics yield higher 
ratings than others? 

c. Do any topics yield shorter 
essays than others? 

d. Do any topics yield longer 
essays than others? 

. 'lOdes 

a. . Do any modes yield lower 

ratings than others? 

b. Do any inoctes yield higher 
ratings than others? 

. c. Do any modes yield shorter 
essays than others? 

d. Do any modes yield longer 
essays than others? 

, Interaction 

a. Does there appear to be an 
interaction of topic and mode? 

. Recowroendatlons 

a. ' Acceptable Topics: 



Circle One 

YES m ? 

YES NO ? 

YES m ? 

YES NO ? 



Cements: 



YES NO ? 

YES NO ? 

YES m ? 

YES NO ? 



YES NO ? 



b. Recommended Mode: 



IX-M-W2 
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RATING 
Accept 



Minor Revisions' 
Major Revisions 
Reject 



-)03- 



TASK II 
MEASUREMENT REVpM 
READING « im 



Passage Code _ 
Reviewers Code 



1. Are the directions clear and 
easy to follow? 

2. Are any dis tractors too close 
In meaning to the keyed answer? 

3. Do the deleted words follow 
the specifications? 

4. Are all the distVacArs for 
each Item- syntactically equiva- 
lent -td the keyed answer? ^ 

5. Are all options for each Item 
'approximately equal In length? 



Circle One 
YES m ? 

YES NO 7 

YES' m ? 

YES NO 7 
YES NO 7 



Coninent if "NO" or "?" 



4 f 



Fonn ii-M-Rl {Use with Item Form Reading 1) 
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RATING 
Accept 



Minor Revisions 
Major Revfslons 
Reject 
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TASK XI 
I^UREfCHT REVIEW 
REAOIlffi - 2 



Passage Code _ 
Reviewers Code 



1. Appropriateness of Format - 

a. Is the format of the Item 
appropriate for the content 
and skill be1n| tested? 

2. Adequacy of Item Stem 

a. Does the stem make clear what 
Is the task to be performed? 

b. ' Is the vfording of the stem 

cU^r? 

c. Is the stem free of extraneous 
or confusing materials?' 

3. I tem Data 

a. Is the difficulty level 
acceptable? 

b. Is the discrimination 
level acceptable? 

c. Are all distractors func- 
tioning? 



" - Circle One 



YES m ? 



Comment If "NO" or "?" 



YES • ? 
YES W ? 
YES NO 7 



YES NO ? 

YES m ? 

YES NO ? 
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ERIC 



^ -orm II-M-R2 



RATING 
Accept 



Minor Revisions 
Major Revisions 
Reject 
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' TASK n 
MEASUREMENT REVIEW 
MATME^TICS 



Passage Code 



Reviewers Code 



1. Appropriateness of Format • 

a. Is the format of the Item 
appropriate fcSr the content 
and skill being tested? 

Z, Adequacy of Item Stem 

a. Does the stem make clear what 
Is the taslc to be parforrod? 

b. Is the wording of the stem 
clear? 

c. Is the stem free of extraneous 
or confusing materials? 

3. Item Data 

a. Is the difficulty level 
acceptable? 

« 

' b. Is the discrimination 
level acceptable? 

c. Are all distractors func- 
tioning? 



Circle One 



YES NO 7 



Cogroent if "NO" or "?" 



YES NO ? 
YES NO 7 
YES NO ? 



YES m 7 
YES NO ? 
YES NO ? 
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. Major ne*<islms 
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TASK n 

mpmom mm 



\vm Cedt 



Rtvlcwtn teds 



1. AppreaHBtww at of Fonmt 

0. Is tht formt of tht Hn 
appropiiot* for tte conttnt 
m4 fkfn being tottedT 

2. AtitKiittev of Itew Stow 

■. Dws the stM Mttai ciMr Mtut 
It ttte task to be pwfrnmdl 

b. If tht wrdlt^ of tl« st«s 
c^eop? 

c. IS thB stM fnw of oxmntegf 
or ronfuf iiHI wtwItU? 

3. A pproprloteraes ' ^tm (^tliww 

a. It titt ke/eb ^brtsmr itatad 
accurately? 

• b. Ar» any other options also 
correct? 

c. Are «\1 the optfoni srannatlcany 
parallel? 

d. Are all the o^k^iens 109<ca11y 
parallel? 

e. Are the options clearly wor^? 

f. Are all options plausible for 
naive examinees? 

f. Are all opt1w»i free of Irrele- 
vant clues? 

fi. Are all options of approxiwtely 
the sane length? 

1. Are the opti«»» reasonably 
ordered? 

a. Does the language of this item 
fellow stanitortf usage? 

b. Is the Hera free of aablgyitles? 

c. An* all the referents explicit 
and correct? 

1, U the iteo free of Bispliced 
nodlfier phrases or claes^? 



Circle One 



YES NO ? 



YES ^ ? 

TES HO ? 

YES m ? 

YES NO ? 

YES NO 7 

YES HO ? 

YES NO Y 



YES m 

YES NO 



YES Iffl^ ? 



YES IB) ? 



YES NO 7 



YES NO 7 



YES no 

YES NO 



YES NO ? 



Form 11 H-P-1 
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nri^T pr«T/ f,'"*M T,!? 
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AcCBPt . 

MijQf RtvUlmi 

ileiect' 



Time II 
HEA^StOCRT itCVIBI 
HtQFSSIOSAl EOUCftTIOl - 2 . 



1. AaprMHiteneM of Hrtmt 

a. It the fonat of th« itm US « t 

a^rosHate for thi cMtwit . , 
•nd skin being tatttd? 

2. AdwMcy of Itaa Stem 

a. OMft tte fttm inako ciMr irtiftt YES NO ? 

1i the Uik to p$Tfvfm«i 

^4^i^mi^t^il^.at^ %Ui^.., . _ )j5 IjP T 

clearT 

€• Is the stem free of extrinem^ ffS HO f 
or omf using natter Uls? 

3. AppropHfttei^ss of ItM Oetioni 

a. 1^ the keyed eniwr ettted »E5 W f 
accurately? 

b. Are any other options also f& m 1 
correct? 

c. Are all the options grasiaatically V£S NO ? 
parallel? 

tf. Are all the options logically t£$ NO' T 
parallel? 

e. Are the options clearly worded? YES l» ? 

f. Are ell options plauslhle for YES NO ? 
nalye exaolnees? 

g. Are all options free of Irrele- YES NO ? 
vant clues? 

h. Ar# all options of approxlnately TE5 m ? 
the sane length? , 

1, Are the optlojis reasonably YES »» ? 

^ oroered? 

4. Item Data 

a. 1$ the <l1ff«CMlty level YES NO 7 
acceptable? 

b. U the discrimination level 1^5 m 7 
acceptable? 

c. Are all dlstractors f*mc- YES MO ? 
tloning? 



Form n 



RATING 
Accept 



Hinor Revisions 
Major Revisions 
Reject 
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. L 



Passage No. 



Reviewers Code 



TASK IX 
BIAS f^IEW 
READIfiG 



1. Does the passage contain any Information 
that could be seen as ojffensive to the 
culture, rarce, or* religion of teacher 
applicants? 

2. Does the passage contain any Infomatlon 
^ - that could be seen as offensive to 

either sexual group? 

3. Does the passage Include ster«>t31)1c ^ 
depictions of any cultural, racial, 

or. religious group that are debasing? 

4. Does the passage Include stereotypic 
depictions of either sejtual group 

that are debasing? • 

5. Does the passage portray cultural, 
racial, or religious groups as 
unequal in ability or natural 
endowments? 

6. Does the passage portray sexual groups 
as unequal In ability or natural en- . 

dowments? 

7. Does the passage contain cli»s or 
infornation that could be seen to -work 
to the benefit or detriment of any 
cultural, racial, or religious group? 

8. Does the passage contain clues or 
information that could be seen to work 
to the benefit or detriment of either 
sexual group? 

9. Does the passage contain group-specifk 
language or vocabulary (e.g., culture- 
related expressions or slang}? 



Circle One 
YES i«D ? 



Coianent If "YES" or 'V 



YES )^ ? 



YES NO ? 



YES m ? 



YES m ? 



YES m ? 



YES 1^ ? 



YES NO ? 



YES NO ? 



©•"Drm II-B-R 
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mm 

' Accept „ 

Minor Revisions 
Major ftevlslORS 
Reject 
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* 



iievi ewers Code 



TASK n 
BIAS ftEVZEM 

vmmN6 



Circle One 

I, Does the 1 ten contain any Information YES Ifi) t 

, that could be seen as offensive to the . 
culture, race or religion of teacher 
applicants? 

.2. Does the item contain any Information . - YES ? 

that could be se^ as offensive to 
either sexual group? 

ft 

^„.4toes-.the.JteBi.1jKliide^r*.emtypi5L..^ , ...... , \0 , J . 

depictions of any cultural racial, 
or religious group that are ctebasing? 

4. Does the Itesi Include stereotypic ' (eS NO 7 
depictions of either sexual group 

that are debasing? 

5. Does the item portray cultural, YES NO 7 
racial, or religious groups as 

unequal in ability or natural 
endowments? 

6. Does the item portray sexual gro^>s YES W) 7 
as unequal In ability or natural 

endowments? 

7. Does the item contain clues or require YES 7 
information that could be seen to wrk 

to the benefit or detrlaent of any 
.cultural, racial, or reHgtous group? 



8, Does the item contain clues or require YES 7 

information that could be seen to work 
to the benefit or detriment of eitter 
sexual group? 



9. Does the Item contain group-specific YES t& 7 

language or vocabulary (e.g., culture- 
related expressions or slang)? ^ 
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•jS-^ II-B-W 



RATINCi 
Accept ' 
Minor RevisloHT 
Majdr Revisions 
Reject 



-no- 



Revleviers Code 



TASK n 
Bm REVIQI 
MATHEMATICS. PTOFESSIOHAL EtHJCATTCHi 

Circle (km 

1. Does the Item contain any Infomatlon YES KO 
that could be seen as offensive to the 
culture, racd» or religion of teadfier 
applicants? 

2. Does the Iteis contain any Inforinatton YES NO 
that could be seen as offensive to 
either sexual group? 

3. Does the item Include stereotypic YES NO 
depictions of any cultural, racial, 
or religious group that are debasing? 

4. Does the Item Include stereotypic YES NO 
depictions of either sexual group 
that are debasing? 

5. Does the Item portray cultural, racial, YES NO 
or religious groups as unequal In 
ability or natural endowments? 

6. Does the 1t«B portray sexual groups as YES f&i 
unequal In ability or natural endoi«»nts? 

7. Does the item contain clues or infbr- YES I© 
mation that could be seen to wrk to , 
the benefit or detriment of any cul- 
tural, racial, or religious group? 

8. Does the item contain clues or Inf&r- YES Jffl 
mation that could be seen tojwrk to 
the benefit or the detriment of either 
sexual group? y 

9. Does the 1t«n contain group-specific YES NO 
language or vocabulary- (e.g., culture- 
related expressions or slang)? 



-m- 

t 
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APPENDIX F 



TEC»#iICAL MATERIALS AND INSTRUCTIONS 
USPD BY 

STANOARD-SEniNG PANEL ^ 



1J3 
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tENTATIVE REOSTOID^TIOre 
COTE 

Jun& 24-26» 1980 
UakulU Springs, Florida 



A recoRiRendatlon of the ptrfiorRiance level which Is to be considered 
"passing" for each of the subt^ts Is provided for consideration. Technical 
. informatdpn will he provl&d to Help you consider end mdlfy each recpm- 

mendatlfsi. 

j|n addition to these materials, you irfll have sas^les of calibrated Items 
• for the three multiple-choice tests and wltlng swales scored at various 

levels. These materials will he1p,y«ui.^t .».fjell0fl,.lqLtJh?.mr^^^ 

level represented by the various s<»res. 

Descriptive data about the field test saaples Is provided in the attached 
table. ^ ' . • 

Tentative Re'coomendatlons 

Reconinendatlon II - It Is recflBwiend^ that the cutting score for the 
Reading Test be set at a point which represents 751 c(»^)rehafis1on on a 
sample of materials from the identified and afk)pted (tomaln. This is a 
logit value of 1.4 idilch will be converted to an RTE scale value of 2(K). 
(This fs close to 80S comprehension and fits thj data.) 

Recomnendatlon #2 - The arinlnal acceptable perfomance level on the 
Mathematics Subtest should be that equal to a logit value of 1.00. 

Recongnendatlon #3 - The winifflal acceptable (^rfprnrance level on the 
Professional Education Subtest should be that equal to a logit value of 
.25. 

Reconinendatiwi 14 - An Individual shall pass the Writing ^ibtest If at 
least two of three trained Judges determine that the writing ^n^le Is 
acceptable; that Is, has a total score of 5 or wore. Thus a rainiimfln 
passing score, based on the sum of thr^ judges' ratings, will be 5. 
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INSTRUCTI8NS FOR READING ITBIS 



You have a sai^le of passages to be used for the Reading 
subtest, arranged % order froa easy to hard . The dif- 
ficulty level is Indicated, with -1.8S being easy and 
3.29 being difficult . 

It Is suggested that you follow these steps: 

(1) "Take" the items as a test. 

(2) Check your answers against the scoring key. 

(3) Sort the passages Into two p^iles: 

A. Applicants should be expected to answer 
most items correctly (all but 1 or 2). 

B. Applicants may miss several Items in 
each passage (3' or more). 

(4) Determine the dividing point (In loglts) 
between the two groups. 

(5) Compare your cutting point with that of 
others in your group and arrive at a 
group decision. 

(6) CoiT?>are the group decision with Tentative 
Recoflmiendatlon §1, 



lis 



' Itecofliiiendatlon #1 - fteading 

Technical MateflaH 

Do nuin of Materials Sjng>led 

The domdin of materials which are sainpled fpr the Reading Subtest were defined as 
follows: » 

Text books: . Textbooks for comnonly r&)u1r8d professional education courses. 
These were identif.ied by a survey of all Rortda institutions with approved 
teacher education prograros. The textbooks wi^ organized into tltese categories: 

1. History, Philosophy, Social Foundations, School Law 

2. Methods 

3. Measuresient 

4. rducational Psychology 

Q% Library Journal^; Non-Specialized educational journals. 

C. State Dgo artn«nt of Education Pi&licatlons ; Selected Departirent of Education 
PMD 11 cations or general Interest. 



Test inn Ttfchnioue 



Multiole-choice cloze test. Test Items constructed by randomly selecting passages 
fro;n the "domain, deleting erfch nth word and providing 4 options from which examinee 
is to selett the correct word. 

Researcn Evidence: 

Refer to artlcia by John R. Bormuth, '•Eii|)1rical Determination of the Instructional 
Readinq Level," International Reading Association CoYiference Proceedings . 1969, 

13, 716-721. 

Suwnary : 

Reading experts^ have traditionally used a criterion of 75% mastery to determine the 
"instructional level" of reading mcterlals. This practice Is based on a recommenda- 
tion oy Thomdike (1917) and widely promilgated by Betts (1946), although there has 
been little empirical evidence to support the practice. 

Somutff conducted a study to investigate the matter. The specific question addressed 
was whether "...there is some range of difficulty which maximizes the amount of 
information students gain as a consequence of readirtg Instructional materials. 

The results of this study suMwrted the use of 75% comprehension as being optimal 
for leamlnq gain. 

Interpretation of the data is complicated by the fact that Bormuth use completion 
CLOZE tests and translated his results into multiple-choice equivalents, based on 
another study. 

11 R 

•article atUched " 
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Normative Infonnation 

The <Jf?tribution of scores for the field- test san^le Is graphed on the attachment. 
Setting the cutting point at a logit of 1.4, which Is equivalent to 80% compre- 
hension, would result In 6.6% falling this subtest. 

READING SIBTEST (50 Items) 
\ DISTRIBUTION OF SCORES 
FIELD TEST - April 19K) 



r 

STATCWlOE 



V>W SCORE 


LOGIT* 


N . 1 3 


CUM. % 


20 & Selow 


- .40 


1 


.08 


.08 


21 


- .30 


1 


.08 


.16 


22 


-.20 


2 


.17 


.33 


■ 22 


- .15 


1 


.08 


.41 


24 


- .10 


2 


.17 


.58 


25 


.00 




• 


.58 


26 


.10 ^ 


4 


.34 


. .92 


?7 


.15 






.92 


:3 


.20 


1 


.Ml 


1.00 , 




.30 


4 


.34 . 


1.34 


:o 


.40 


2 


. 17 


1.51 




50 


5 


* .42 






.60 


3 


.25 


2.18 


33 


.70 


S 


.42 


2.60 


34 


• .75 


4 


.34 


2.94 


35 


.80 


7 


.59 


3 • S3 


:6 


.90 


4 


.34 


3.87 


37 


1.00 


6 


.51 




38 


/ 1.10 . 


11 


.93 


5-31 


39 


1.20 


ft 


1.26 


6.57 


40 


1.40 


37 


3.12 


9-69 


41 


1.50 


32 


2.70 


12.39 


42 • 


1.65 


54 


4.55 


16.94 


43 


1.80 


55 


4.64 


21.58 


44 


2.00 


79 


6.66 


28.24 . 


S5 


2.20 


107 


9-02 


37.26 


46 


Z.XO 


139 


11.72 


48.98 


47 


2.80 


167 


14.08 


G3.06 


4S 


3.20 


162 


13.66 


76.72 


49 


3.60 


180 


15.18 


91.90 


50 


4.00 + 


96 


8.09 


99.99 




4 


1.186 










V 




•From Rdsch Analysis 












' 1 
1 1 


i / 





R E A D I fl G 



.-STATEHinE. 




V^it '^iM'f^y^ ?] 72 24 25 26 27 Z« 29 30 31 32 3 3 31 35 M 37 311 3« 40 41 42 43 44 45 46 47 4<» 4Q 50 

fiH.II -,4« ..2U -.10 to ,..20 _ .40 ^„ .60,^ .7b ^ .W . 1.10 1.4u 1.65 2>) 2.40 3.20 4.00> 
- '^ '5 .30 . 50 .70 . 80 1.00 1.20 1.50 l.BO / 2.20 ?.4o .1.60 

f 
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IHSTRUaiONS FOR MTtiEHATICS ITEMS 



You have a sample of mathenatlcs Itens, arranged In order from 
easy to hard . The difficulty leyel Is indicated, with -1.30 
being easy and 1.50 being difficult. 

It is suggested that you follow these steps: * 

(1) "Take" th^ items as a test. 

(2) Check your answers against the scoring key. 

(3) Sort the itens Into two piles: 

A. Applicants should be.exp^ted to answer 
correctly more than half cf Iters of this 
type. 

B. Applicants should be expected to answer 
correctly less than half of Items of this, 
type. 

(4) Determine the dividing point (in logits) between 
the two groups. 

(5) Compare your cutting iK)int with that of others in 
your group and arrive at a group declFlon. 

(6) Compare the group decision with Tentative 
RecomnendBtion #2. 
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ItecoRiiieidatlofl 12 - Nathowtlcs 
Technical HateHals 



Description of Test Items 

The examinatloa covers only basic nathematlcsi with very little Algebra or other 
higher mathematics. Hair the Iteiss are sliaple computation or one-step word 
problems, the other half are "real world" probletns. 

The Items are multiple choice, and only 36$ Include "None of the Above" as an 
option. 

In general, the Items are judged to be fairly ^easy." 



Normative Information 

The distribution of field-test scores Is quite skewed. For the 27 Ite^, the 
mean was approximately 22; SD was approximately S. That means that 2/3 of the 
scores fell between 17 and 27. The logit for 17 Is .50. The "break" In the dis- 
tribution comes between 19 and 20, with a logit value of l.(K). 

Setting the cutting score at .90 will result In approximately 30% failing, A 
cutting score of .50 would fall 161. 

♦ * 

Additiona l Information ^ 

The "passing" score for student assessnwnt for similar Items Is 70S. For 27 Items, 
that would be a raw score of 19, with a logit value of .80. 
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miHeiATICS SIBTEST (27 Items) 
" DISTRIBUTION OF SCORES 
FIELD TEST - April 1980 





LOGIT* 


11 


STATEWIDE 


CM % 


V 




16 


1.4 


1.4 


ft 




8 


.7 


2.1 


q 




ft 


.7 


2 8 


10 




16 


1.4 


4.2 




• .65 


13 


1.1 • 


5.3 

www 


12 


- .40 


18 


1.5 


6.8 


n 


- .20 


17 


1.4 


8.2 


14 


- in 

P AW 


21 


1.8 


10.0 


15 


.10 


37 


3.1 


13.1 


16 


.3d 


36 


3.0" 


16.1 


17 


.50 


54 


4.6 


20.7 


18 


.60 


53 


4.5 


25.2 


19 


.80 ' 


:7 


4.8 


30.0 


''O 


1.00 


78 


6.6 


36.6 


21 


1.30 


87 


7.3 


43.9 


22 


1.50 


88 


7.4 


51.3 


23 


1.70 


122 


10.3 


61.6 


24 


2.00 


145 


12.2 


73.8 


25 


2.40 


142 


12.0 


85.8 


26 


2.90 


106 


8.9 


94.7 


27 . 


3.60 


64 


5.3 


lOO.O 






i.ise' 







• From Rasch Analysis 
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m CFMT 

17.01 

16.5 

16.0 

15.5 

15.0 

M.S 

14.0 

13.5 

13.0 

12.5 

12.0 

11.5 

n.o 

10.5 
10.0 
9.5 
9.0 
d.S 
8.0 
7.5 
7.0 
6.5 
6.0 
, 5.5 
5.0 
4.5 
4.0 
3.5 
3.0 
2.5 
2.0 
1.5 
1.0 
.5 



1 , , 
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-RAW SCORE 7 8 



n 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 



ERIC 



-1.40 -1.00 -.65 -.20 .10 ^ •SO .8C 1.30 1.70 2.40 3.60 
-1.20 -.80 -.40 -.10 .») .60 I.CHI 1.50 2.00 2.90 
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Reconssendatlon 13 
Technical Haterials 

DescHptlon of the Test Item 

a. The test will consist of 100 ntultlple-ctoice Hems. 

b. The test is geneHc; that is, it covers geieral principle^^ and recotmsnded 
procedures, rather than those related to specialized areas. 

,c. Host of the items have been tiritten at the application or problea-solving 
level. This was done in order to provide a test task whicn is as close to 
actual classroom situations as possible with a paper and penci} test. 

d. In writing application and problew-solving items, a scenario is frequently 
used to. present a context imich specific ii^rtant, relevant varii .es. 
There are two potential prc^leis associated with this practice: 

1. Some deqree of anhiguity my result since it is not always possible to 
specify all relevant variables. 

2. The context -chosen may not be familiar to a given eiominee. 

The Bultiple review process^ have aiMressed tim. first probl^ to sone extent; th6 
second has been handled by deliberately vaiylng context sc that the>ie is a balance 
between various grade levels and subJ^iU. this seans that eveiy exacinee will have 
some items for which the context na^ be tsifamillar, em thotptb principle being 
tested is one which should be faoriliar to all tocher ai^licants. (See attached analysis.) 

TUe correctness of the keyed response can be established either by reference to 
professional literature or by expert opinion or both. Experts differ to some 
extent as to the correctness of the keying of the items. Application and problem- 
soivi nn require the Integration of many issues and theories, so the correct answer, 
1?lieldom clear-cut. The keyed answer is the one selected roost frequently by 
"experts." but the percent of experts choosinf'the keyed answer varies from one 
item to another. 

Horwative Information 

Except for a small percentage of cases with scores at or near the chance level, 
score on the Professional Education- Subtest are almost normally distributed, with 
a mean of approximately 34, SD of approximately 6. The modal score Is 36, with 38 
being the- next most frequent score. 

The distribution has a slight "break" between a logit of .40 and .45. Setting the 
cutting score at .45 would fail 2016. 

Raw scores lower than 25, made by 9% of the sample, probably have a large component 
of chance. 

The recomnended cutting point of .25 -lies at -1 SD from the mean and above. the 
50:; chance level- The percent failing with this cutting score would be 13S. 
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iNSTRucTicms m mmsimi edik:atiom items 



You have a sai^le of Items for the Frofesslonal Education 
subtest, arranged In order f«» easy to hard . The difficulty 
level Is Indicated. iHth -1.60 beini easy and 2.10 being 
difficult. ^ . 

It Is suggested that you fo11oi»r these steps: 

(1) -Take" the Items as a test. 

(2) Check your answers agafnst the scoring key. 

(3) Sort the Itens Into two piles: 

A. Applicants should be expected to answer 
correctly more than half of Items of this 
type. 

- B. Applicants should be expected to answer cor- 
rectly less than half of Items of this type. 

(4) Determine the dividing point (in loglts) between 
the two groups. 

(5) Conipare your cutting point with that' of others In 
your group and arrive at a group decision. 

(6) Compare the group decision wjth Teitatlve Recorauen- 
datlon 13. 
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PROFESSIOHAl EWfCATION SUBTEST 
DISTRIBUTION OF SCORES 
FIELD TEST - April 1980 







StATtWite 






LOGIT* 


N 


% 


C\M. % 


& 3elQw 


-1.35 


6 


.5 


.5 


13 


-1.27 


1 


.1 


.6 


14 


-1.10 


3 


.3 


.9 


IS 


-l.(K) 


4 


.4 


1.2 


16 


- .95 


10 


.9 


2.1 


17 


- .K) 


4 


.4 


2.5 


18 


- .65 


8 


.7 


3-2 


19 


- .60 


2 


.2 


3.3 


20 


- .50 


7 


.6 


4.0 


2! 


- .40 


9 


.8 


4.7 


22 


- .30 


11 


1.0 


5.7 


23 


- .20 


10 


.9 


6.6 


24 


- .16 


14 


1.2 


7.8 


35 


.00 


17 


1^5 


9.3 


26 


.05 


19 


1.7 


11.0 


27 


.20 


27 


2.4 


13.3 


23 


.25 


31 


2.7 


16.1 


29 • 


.40 


43 


3.8 


19.8 


30 


.45 




3.2 


23.0 


31 


.50 


55 


4.8 


27.8 


22 


.70 


59 


5.2 


33.0 


33 


.80 


66 • 


5 8 


38.8 


V"* 


.92 


79 


6.9 


45.7 


35 


1.00 


78 


6.8 


62.5 


36 


I.IO 


104 


9.1 


61.6 


37 


1.20 . 


73 


6.4 


68.0 


33 


1.30 


93 


8.2 


76.2 


39 


1.50 


72 


6.3 


82.5 


40 * 


1.60 


61 


5.3 


87.8 . 


41 


1.70 


46 


4.0 


9t.9 


42 


2.00 


26 


2.3 


94.1 


13 


2.10 


32 


2.8 


• 96.9 


44 


2.40 


20 


1.4 


96.7 


45 


2.60 


9 


.8 


99.5 


2o 


2.'70 


3 


.3 


99.7 ' 


47 


3.20 


3 


.3 


100.0 






1,141 







♦From Rasch Analysis 
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HAU SIORK 1? 13 14 IS . IS 17 18 19 20 21 22 23 ?4 2S 26 27 28 29 30 31 32 33 34 35 36 37 38 19 40 41 4? 43 44 4S 46 47 



:!.35 -1.10^ -.95 -.65 -.50 -.30 ,jg ,zq *.4o .50 .80 1.00 1.20 ' 1.60 2.00 2.40. 2.70 
-1 27 -1,00 ..80 -.60 -.40 -.20 .00 .25 . 45 , 70 ;92 1. 10 1.30 1.70 2.10 2.60 3.20 
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oiREai(»ts FOR mum s/m.Es 



You have elevan Miitlns sanples, selected to represent a wide 
range of scores assigned fay trained Judges. Soise are "Unac- 
ceptable" (score of 3)* soae are "Nlnlioally acceptable" 
(score of 5 or 6). others are "Acc^table" (score of 7-9) or* 
"Superior" (score of 10-12) 

It 1s^s«««t«. that ,ouf.l1« these .teps: 

(I) Sort the essi^ Into 4 piles* acaii^lng to 
your judgnmt as to overall quality* using the 
scoring criteria fHr^aited 1 n Attadmit 1 . 
Your groups should be: 

Unacceptable (Rating of 1 



Mlnlinally acceptable 
Acceptable 
Superior 



Hating of 2 
Rating of 3 
Rating of 4] 



(2) Check your sorting against the Judges' scores, 
using Table w-1. 

r 

(3) Consi<ter whetter the scoring directions, as 
written In Attachment 1, correctly classify 
the essays. 
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Rea»imdat1on #4 - Writing 
Technical Materials 

DescHptlon of the Test > 

The Writing Sid>t8St will be a product1<»i uniting s^le scored by three trained 
and experienced judges, with dlscr^ancles rssotved by a "referee." 

The standard for passing Is built Into the scoring standard. (See attachn^t 1.) 

Preliminary studies have Indicated that judges can be trained to attain very high 
reliabilities - better than 90% agreaaent. 



Normative Information 

The Writing Subtest was not Included fn the field test. Hoitever, an earlier study 
was carried out which provided S09« data. In the fall of 1979, writing sauries 
w^re collected from 3fiO students at two teacher training Institutions, Florida 
Scate University and University 0f South Florida. The sai^ile Is not completely 
representative of the population becai^e certain ettmic and progrm groups were 
undur- represented. The description of the sa^ile and the distribution of scores 
is attached. 

Because of the non-representativeness of tue sai^le. It should be anticipated that 
the percent failing the Writing test will be higher than the lOSt found in the 
study, perhaps being as high as 20«. 



Scoring Procedure Recommeided for the Writing 
Test by the Contractor 

Scoring System to be Used 

The scoring system for evaluating essays written on the Florida Teacher Certifica- 
tion Examination is as follows: each may will be read 1nd^>end«ntly by three 
raters, who will assign It a rating of 1, 2, 3, or 4, Spending on his or her judge- 
n^nt of Its overall quality. (The meaning of tliese ratings Is discussed later.) If 
one of the three ratings differs frow any other by two points or. more, or If the 
three ratings are some cor*1nat1on of I 1 2, the ffflper will be sent to a. fourth rater, 
or refen?e, who will read it and assign It a rating. (This rating will then replace 
the most discrepant one.* The sum of three ratings for ^ch paper— either the three 
original ones, or two originals and a referee's substitute— Is Its final score. The 
range of scores for each essay Is thus 3 at the low end to 12 at the top.) Note that 
no paper Will have a final score of 4. A minimally passing score is 5. 



♦If the referee's rating is' 1, then It Veplaces the 2. so the score becon^ 3. If 
it is 2 or higher, it replaces a 1, so the score becomes 5 or higher. 
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OescHptlm of Ssi^le for Writing Study 



FSU 



Hajor 

Hiyslcal Ed. 
S{^1a1 Ed. 
Husic £d. 
Elesen^ry Ed. 
English Ed. 
Speech. Pathology 
Social Studies Ed. 
Hone Economics Ed. 
Art Ed. 

Katheiiiatlcs Ed. 
Social Work 
Early Oilldhood Ed. 
Child Developneit 
Science Ed. 
Visual Disabilities 
Library Science 
Vocatlonal/Suslness 
Industrial Arts Ed. 
Foreign Language Ed. 
Theater Ed. 
Political Science 
ESL 

Psychology 
Art Therapy 
Career Ed. 



Ed. 





USF 




# In'Smle 


fiajpr 


# in SaRS)le 


5S 


• 

Elmntary Ed. 


9a 


21 


Early Chllittiood Ed. 


36 


17 


Learning Disabilities 


16 


17 




8 


13 


m 


6 


10 


English Ed. 


2 


7 


Gifted Ed. 


2 


6 


Forelm Languai^ Ed. 


1 


5 


Deaf Ed. 


1 


5 






5 






4 






4 






3 






3 


- — - 


•■ 


3 






2 






2 






2 






1 






1 






1 






1 






1 






1 






m 







Distribution of Scores on Writing Senile 



Score 


N 


X Of N 


3 


37 


10.3 


*4 


0 


0 


5 


100 


27.8 


6 


71 


19.7 


7 


76 


21.1 


8 


47 


13.1 


9 


13 


3.6 


10 


13 


3.6 


n 


2 


.6 


12 


1 


.3 


Total 


360 


100 



Hean Score— 6.19 
Median Score"6 
Modal Score— 5 



•Papers with a score of four mte reviei«d and the scores were adjusted to either 
a three or a five depending on the referee's rating. / * 
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The Rating Criteria for the MHtlna Si^test 

The criteria were developed to accofmiodate three specific conditions ln^osed by the 
writ! fig test: 

<■ ' 

I. that ^hey reflect those characteristics which are irtdely accepted as indicatlvis 
of good writing and which can thus reasonably be expected to appear in essays 
written by people. seeking Initial Florida Teacher Certification; 

2/ that they arfr amenable to »be1ng "translated" 'into operational descriptions of: 
levels of con^tence that can be used by raters; 

3. that they Uke Into account the writing stibskllls Identified by the Council 
6*1 Teacher Education. 

1 

Criteria to be Used by Judges for 
the Evaluation of Essays 



1. Rhetorical Quality 

^1.1 Unity: An ordering and Interdependence of parts producing a 
single effect: completeness. 

1.2 Focus: Concentration on the chosen topic. 

1.3 Clarity: Lucidity of expression; lack of antlgulty and distortion. 

1.4 Sufficiency: Appropriate depth and breadth of expression to meet 
the writer's purposes and the demands of the particular topic. 



2. Structural and Mechanical Quality 

2.1 Organization: Consistent and coherent Integration and connection 
of parts. # . 

2.2 Developnent: Appropriate and sufficient exposition of ideas; \use 
of detail, examples* illustrations, comparisons, etc. 

2.3 Paragraph and Sentence Structure: Appropriate form, variety, logic, 
relatedness of and anrang structural units. 

2.4 Syntax: Appropriate ordering of v/ords to" convey intended meaning. 



3, Observance of Conventions In Writing 

3.1 Usage: Appropriate use of language features: inflections, tense, 
agreement, pronouns, nsjdifiers, vocabulary, level of discourse, etc. 

3.2 Spelling. Capitalization, Punctuation: Consistent practice of accepted 
forms. 
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THE RATIHS SCALE 

The four-level scale Is used because It provlites enough degrees of distinction to 
meaningful In assessing writing coaipetence. 



1. The essay lacks unity and focus. It is distorted and/or ambiguous, 
and it fails to treat the topic In sufficient depth and breadth. 
There is little or no discernible organization and only scant 
development of Ideas, If aqy at all. The essay betrays only 
sporadically a sense of paragraph and s«»tence structure, and 

it is syntactically slipshod. Usage is Irregular and often 
questionable or wrong. There are serious errors In spelling, 
capitalization, and punctuation. 

2. The essay has sane degree of unity and focus, but each could be 
improved. It Is reasonably clear, though not Invariably so, and 
it .treats the topic with a marginal degree of sufficiency. The 
essay reflects some concern for organization and some for 
development of ideas, but neither Is necessarily consistent nor 
fully realized. The essay reveals s(w« s«»se. If not full 
command, of paragraph and sentence structure. It is syntactically 
bland and at times, awkward, l^age is generally accurate, if 

not consistently so. There are some errors in spelling, capi- 
talization, and punctuation that detract from the essay s effect 
if not from Its sense. 

3. The essay Is focussed and unified, and It Is c1«arly l^' jot 
distinctively written. It gives the topic an adequate though not 
always thorough treatment. The essay is well organized, and ^ 
must of the tiwe It develops Ideas appropriately and sufficiently. 
It shows a good grasp of paragraph. and sentence structure, and 

its usage Is generally accurate and sensible. Syntactically, 
it is clear and reliable. There may be a few errors in spelling, 
capitalization, and punctuation, but th^ are not serious. 

4. The essay is wiified, sharply focussed, and distinctively effective. 
It treats the topic clearly, completely, and in suitable depth 

and breadth. It is clearly and fully organized, and It develops 
ideas with consistent appropriateness and thoroughness. The essay 
reveals an unquestionably firm command of paragraph and .sentence 
structure. Syntactically, it is si«>oth and often elegant;. Usage 
is uniformly sensible, accurate, and sure. There are ve»y few, 
if any, errors In spelling, capitalization, and punctuation. 
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I. Professional issws 

Berliner, D. and Tikunoff, W. The CallfomU Beginning Teacher Evalwtlon 
Study: Overview of the Ethnographic StiK^y. ^toumal of Teacher Education , 
Spring, 1976 

Oage, N. U, Ed., Han^flxfflk of Research on Teaching, Chicago: Hand McNally, 
1973 



McNeil, J. D. and Po|*aw. M. 0. The AssessffliTt of Teacter Oraptence, 
Seccwd Hantfeook of Research on Teaching, R. H, Trawrs, Ed., «»r1can 
Educatl^mBlResearch Association* Qilca^: Rand Hdtolly, 1973 



Medley, D. M. Teacher Ceapetaira and Tead ^r Effectiii«B^; A tevlew of 
Process - Product Resfflrch. Washington, P.C,: ftaerican Association of 
Colleges for Teacher EdiKatioii, 1977 

Peterson, D. and Hard, A., Ed., U*^ ^^^V^ "^gggl®^ Evaluatiw. 
Washington, D.C.: University Press of America, im> 

Popham, W. J. Perfo?i8ance Tests of T^chlng Profideniaf: Rationale, 
Development, and Validation, Awerican tAicational Res earch Journal, 
June, 1971 

Rosenshine, B. and Furst, N. Researdi on Teacher Perfbnsance Criteria in 
Research in Teacher Education; A Syiaws iip, B. Othanel Snith, Ed., 
rnglewood Cliffs; N J: Pr^tice-«al1. i^h 

II. General Technical Issues 

Bloom, B. S. Learning for Masteiy . HandbSok tm Fomatlve and Suroaative 
/ Evaluation o f Student Learning. B. S, Blo<»i, J. T. Hasting, and 

/ 6. F. Madaus. fedb.. Sew Yo,k: Mcfiraw-Hill, 1971 

Hambleton, R. K. and Ei^r, D. R. Guidelines for Evaluatii^ 
Referenced Tests and Test Manuals. Journal of Educational Research, 
1978, 15 

Hambleton, R. K. and Novick, M. R. Toward an Integration of JJeo^e^lln.^* 
Methods for Criterion-Referenced Tests. Journal of E ducational Measureinent, 
1973, 10. 159-170 

t 

Standards for Educational and Psychological Tests. Washington, D.C.: 
American Psychological Association, 1974 

Thomdike, R. L. (Ed.) Educational Weasurement , Second Ed., Washington, 
D.C.: Afflerican Ocfcincil on Education, 1971 
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